Jul 16 04:48:17.074958 ip-10-0-138-249 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 04:48:17.074967 ip-10-0-138-249 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 04:48:17.074974 ip-10-0-138-249 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 04:48:17.075308 ip-10-0-138-249 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 04:48:27.240004 ip-10-0-138-249 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 04:48:27.240020 ip-10-0-138-249 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d7f23fba23eb4b7f94a9fff232e7f919 --
Jul 16 04:50:02.919488 ip-10-0-138-249 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 04:50:03.389674 ip-10-0-138-249 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 04:50:03.389674 ip-10-0-138-249 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 04:50:03.389674 ip-10-0-138-249 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 04:50:03.389674 ip-10-0-138-249 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 04:50:03.389674 ip-10-0-138-249 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 04:50:03.390972 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.390886    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 04:50:03.397389 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397370    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 04:50:03.397389 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397386    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 04:50:03.397389 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397391    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 04:50:03.397389 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397396    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397400    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397405    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397409    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397412    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397416    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397420    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397423    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397427    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397430    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397442    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397446    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397449    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397453    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397457    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397461    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397465    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397469    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397472    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397476    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 04:50:03.397625 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397480    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397486    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397493    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397497    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397501    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397506    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397510    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397514    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397518    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397522    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397527    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397530    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397534    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397538    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397543    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397547    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397552    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397556    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397560    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397564    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 04:50:03.398237 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397569    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397575    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397580    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397585    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397589    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397594    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397598    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397602    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397606    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397610    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397614    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397618    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397622    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397626    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397630    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397636    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397640    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397644    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397648    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 04:50:03.398710 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397653    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397657    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397661    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397665    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397669    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397673    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397678    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397683    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397687    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397691    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397696    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397701    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397705    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397709    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397713    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397717    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397722    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397730    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397734    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397738    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 04:50:03.399161 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397742    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397747    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397751    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397755    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397759    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397763    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397767    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397771    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397776    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397780    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397784    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397787    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397794    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397799    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397804    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397808    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.397813    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398664    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398675    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398679    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 04:50:03.399601 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398682    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398684    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398687    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398690    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398692    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398695    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398697    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398699    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398703    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398705    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398708    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398710    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398713    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398715    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398718    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398720    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398723    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398725    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398728    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398730    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 04:50:03.400118 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398732    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398735    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398744    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398746    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398749    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398757    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398759    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398762    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398765    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398767    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398770    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398773    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398775    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398778    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398780    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398782    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398785    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398787    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398790    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398792    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 04:50:03.400856 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398795    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398797    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398800    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398802    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398804    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398807    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398809    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398811    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398814    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398816    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398819    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398821    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398824    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398847    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398851    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398860    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398863    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398865    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398869    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 04:50:03.401491 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398874    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398877    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398879    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398882    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398885    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398887    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398889    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398892    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398895    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398898    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398900    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398902    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398904    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398907    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398909    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398911    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398914    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398916    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398919    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 04:50:03.401944 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398922    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398926    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398930    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398933    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398935    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398938    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398940    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398943    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398946    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398949    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398957    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398960    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398962    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398965    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398967    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398970    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398972    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.398975    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399417    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 04:50:03.402391 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399422    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399425    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399428    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399431    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399433    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399437    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399441    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399443    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399446    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399448    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399451    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399453    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399455    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399458    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399460    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399462    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399465    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399467    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399469    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 04:50:03.402847 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399471    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399474    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399478    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399481    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399490    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399493    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399495    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399498    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399501    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399503    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399506    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399508    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399511    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399513    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399515    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399517    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399520    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399522    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399524    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 04:50:03.403282 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399527    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399530    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399532    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399534    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399537    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399539    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399541    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399543    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399546    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399548    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399550    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399552    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399555    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399557    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399559    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399562    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399564    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399567    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399574    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 04:50:03.403714 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399577    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399581    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399584    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399587    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399590    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399593    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399595    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399598    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399601    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399603    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399607    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399609    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399613    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399616    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399618    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399620    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399623    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399625    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399627    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399630    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 04:50:03.404173 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399632    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399634    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399636    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399639    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399641    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399643    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399646    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399648    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399651    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399653    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399655    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399658    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399665    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399669    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399671    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399674    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399676    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399679    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399681    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399684    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 04:50:03.404624 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.399686    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399755    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399761    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399771    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399775    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399779    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399782    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399786    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399790    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399793    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399796    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399799    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399802    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399805    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399808    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399811    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399813    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399816    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399819    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399838    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399842    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399845    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399848    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399851    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 04:50:03.405113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399854    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399863    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399866    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399869    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399872    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399875    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399878    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399881    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399884    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399888    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399891    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399893    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399910    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399913    2565 flags.go:64] FLAG: --enable-server="true"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399916    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399925    2565 flags.go:64] FLAG: --event-burst="100"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399928    2565 flags.go:64] FLAG: --event-qps="50"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399931    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399934    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399936    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399940    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399943    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399946    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399948    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399951    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 04:50:03.405646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399954    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399956    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399959    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399962    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399964    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399967    2565 flags.go:64] FLAG: --feature-gates=""
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399970    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399973    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399976    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399985    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399988    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399990    2565 flags.go:64] FLAG: --help="false"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399993    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399996    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.399998    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400001    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400004    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400007    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400010    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400012    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400015    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400017    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400020    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400023    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 04:50:03.406338 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400026    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400029    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400031    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400034    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400036    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400039    2565 flags.go:64] FLAG: --lock-file=""
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400042    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400044    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400047    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400052    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400055    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400058    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400060    2565 flags.go:64] FLAG: --logging-format="text"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400063    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400066    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400069    2565 flags.go:64] FLAG: --manifest-url=""
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400072    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400076    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400088    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400096    2565 flags.go:64] FLAG: --max-pods="110"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400099    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400102    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400104    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400107    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400110    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 04:50:03.406924 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400113    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400115    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400127    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400130    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400133    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400136    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400139    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400143    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400146    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400149    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400151    2565 flags.go:64] FLAG: --port="10250"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400154    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400157    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-096732d8e8b333753"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400160    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400162    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400165    2565 flags.go:64] FLAG: --register-node="true"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400168    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400175    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400178    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400180    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400183    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400186    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400189    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400192    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400195    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 04:50:03.407510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400197    2565 flags.go:64] FLAG: --runonce="false"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400247    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400462    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400473    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400480    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400485    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400491    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400498    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400504    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400511    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400522    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400526    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400531    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400536    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400541    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400546    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400562    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400567    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400574    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400598    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400602    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400607    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400612    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400617    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400622    2565 flags.go:64] FLAG: --v="2"
Jul 16 04:50:03.408120 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400635    2565 flags.go:64] FLAG: --version="false"
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400641    2565 flags.go:64] FLAG: --vmodule=""
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400647    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.400652    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401553    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401564    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401568    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401571    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401575    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401577    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401585    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401589    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401591    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401594    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401596    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401599    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401601    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401604    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401606    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401609    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 04:50:03.408691 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401611    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401613    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401616    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401618    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401621    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401623    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401626    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401628    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401631    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401633    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401636    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401638    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401640    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401644    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401648    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401650    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401653    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401656    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401659    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401662    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 04:50:03.409183 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401664    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401666    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401671    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401674    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401677    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401679    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401682    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401684    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401687    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401689    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401692    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401694    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401696    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401698    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401701    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401703    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401706    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401708    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401711    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 04:50:03.409671 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401713    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401715    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401718    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401720    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401722    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401725    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401727    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401730    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401732    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401734    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401738    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401740    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401743    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401746    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401748    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401750    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401753    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401755    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401758    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401760    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 04:50:03.410137 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401763    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401765    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401768    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401770    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401773    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401775    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401777    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401780    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401782    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401784    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401786    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401789    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401791    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401794    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401796    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401798    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401801    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401803    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401805    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401808    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 04:50:03.410598 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401810    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 04:50:03.411062 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401813    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 04:50:03.411062 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401816    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 04:50:03.411062 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.401819    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 04:50:03.411062 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.402612    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 04:50:03.411062 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411045    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 04:50:03.411062 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411059    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411081    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411087    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411140    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411145    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411148    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411151    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411154    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411157    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411159    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411162    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411164    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411167    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411169    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411171    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411174    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411176    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411179    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411181    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 04:50:03.411209 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411185    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411187    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411190    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411192    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411195    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411197    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411199    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411202    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411204    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411206    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411209    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411211    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411214    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411218    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411222    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411225    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411228    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411230    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411233    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 04:50:03.411643 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411235    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411237    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411240    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411242    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411245    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411247    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411249    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411252    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411254    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411256    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411258    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411261    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411263    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411267    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411271    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411276    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411279    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411281    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411284    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 04:50:03.412134 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411287    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411289    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411292    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411294    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411296    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411299    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411301    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411304    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411306    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411308    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411311    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411313    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411315    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411318    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411320    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411323    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411325    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411327    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411329    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411332    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 04:50:03.412556 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411334    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411336    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411339    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411342    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411345    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411348    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411350    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411352    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411356    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411358    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411361    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411363    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411365    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411367    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411370    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411372    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411375    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411377    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411379    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411381    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 04:50:03.413021 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411384    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411386    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411388    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411391    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411393    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411398    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411506    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411510    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411512    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411515    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411518    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411520    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411522    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411525    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411527    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 04:50:03.413472 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411529    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411532    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411534    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411536    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411539    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411541    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411544    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411547    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411549    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411551    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411554    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411556    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411559    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411561    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411563    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411567    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411570    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411573    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411576    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 04:50:03.413817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411579    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411582    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411585    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411587    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411589    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411591    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411594    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411596    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411598    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411602    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411604    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411607    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411610    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411612    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411614    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411617    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411619    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411621    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411624    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411626    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 04:50:03.414266 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411629    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411632    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411634    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411636    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411639    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411641    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411643    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411646    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411648    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411651    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411653    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411655    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411658    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411660    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411663    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411665    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411667    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411669    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411672    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411674    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 04:50:03.414730 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411676    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411678    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411681    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411684    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411686    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411688    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411690    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411693    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411695    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411698    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411700    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411702    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411704    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411708    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411710    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411712    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411714    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411717    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411719    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411721    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 04:50:03.415208 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411723    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411726    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411728    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411730    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411732    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411735    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411737    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411739    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411742    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411744    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:03.411746    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411751    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.411869    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 04:50:03.415656 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.414245    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 04:50:03.416101 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.416037    2565 server.go:1019] "Starting client certificate rotation"
Jul 16 04:50:03.416155 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.416138    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 04:50:03.416182 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.416173    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 04:50:03.419389 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.419377    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 04:50:03.420571 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.420555    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 04:50:03.431593 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.431573    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 16 04:50:03.433978 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.433956    2565 log.go:25] "Validated CRI v1 image API"
Jul 16 04:50:03.434814 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.434801    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 04:50:03.439277 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.439245    2565 fs.go:135] Filesystem UUIDs: map[1b8b2816-5c1d-40a3-a3a4-47e9c83b4c12:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ca0e677b-fb14-4e5f-913b-1c0ce4c76933:/dev/nvme0n1p3]
Jul 16 04:50:03.439352 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.439275    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 04:50:03.443180 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.443163    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 04:50:03.444240 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.444133    2565 manager.go:216] Machine: {Timestamp:2026-07-16 04:50:03.442728108 +0000 UTC m=+0.363310893 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200847 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e7cfdb6b8285ab337c933bf4227c8 SystemUUID:ec2e7cfd-b6b8-285a-b337-c933bf4227c8 BootID:d7f23fba-23eb-4b7f-94a9-fff232e7f919 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:54:6a:be:61:9b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:54:6a:be:61:9b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:1f:63:9c:2f:6e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 04:50:03.444240 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.444235    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 04:50:03.444334 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.444313    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 04:50:03.447081 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.447056    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 04:50:03.447208 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.447085    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-249.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 04:50:03.447921 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.447911    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 04:50:03.447954 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.447924    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 04:50:03.447954 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.447931    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 04:50:03.448724 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.448715    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 04:50:03.450556 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.450546    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 04:50:03.454879 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.454867    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 04:50:03.454999 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.454989    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 04:50:03.455029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455020    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 04:50:03.455055 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455029    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 04:50:03.455055 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455043    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 04:50:03.455055 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455053    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 16 04:50:03.455166 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455061    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 04:50:03.455418 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455407    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 04:50:03.455460 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.455425    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 04:50:03.458022 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.458001    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 04:50:03.460456 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460442    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 04:50:03.460519 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460472    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 04:50:03.460694 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460684    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 04:50:03.460723 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460700    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 04:50:03.460723 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460718    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 04:50:03.460802 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460726    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 04:50:03.460802 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460738    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 04:50:03.460802 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.460746    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 04:50:03.461291 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.461282    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 04:50:03.461320 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.461300    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 04:50:03.461320 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.461307    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 04:50:03.461320 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.461312    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 04:50:03.461320 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.461321    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 04:50:03.461421 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.461328    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 04:50:03.462171 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.462162    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 04:50:03.462201 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.462172    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 04:50:03.465422 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.465403    2565 server.go:1268] "Started kubelet"
Jul 16 04:50:03.465518 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.465499    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m98sh"
Jul 16 04:50:03.465518 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.465493    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 04:50:03.465611 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.465545    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 04:50:03.465671 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.465611    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 04:50:03.466277 ip-10-0-138-249 systemd[1]: Started Kubernetes Kubelet.
Jul 16 04:50:03.467289 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.467271    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 04:50:03.467740 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.467726    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 04:50:03.470105 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.470068    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 04:50:03.470187 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.470081    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-249.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 04:50:03.470328 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.470288    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 04:50:03.471758 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.471738    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 04:50:03.472927 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.472901    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m98sh"
Jul 16 04:50:03.474250 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.471784    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-249.ec2.internal.18c2abe53dd51cde  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-249.ec2.internal,UID:ip-10-0-138-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-249.ec2.internal,},FirstTimestamp:2026-07-16 04:50:03.465374942 +0000 UTC m=+0.385957726,LastTimestamp:2026-07-16 04:50:03.465374942 +0000 UTC m=+0.385957726,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-249.ec2.internal,}"
Jul 16 04:50:03.474363 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.474269    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 04:50:03.474363 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.474267    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 04:50:03.474456 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.474362    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:03.475108 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475054    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 04:50:03.475108 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475076    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 04:50:03.475108 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475089    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 04:50:03.475276 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475183    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 04:50:03.475276 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475191    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475450    2565 factory.go:153] Registering CRI-O factory
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475466    2565 factory.go:223] Registration of the crio container factory successfully
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475516    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475525    2565 factory.go:55] Registering systemd factory
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475532    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475562    2565 factory.go:103] Registering Raw factory
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475575    2565 manager.go:1196] Started watching for new ooms in manager
Jul 16 04:50:03.479024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.475957    2565 manager.go:349] Starting recovery of all containers
Jul 16 04:50:03.480763 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.480718    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 04:50:03.480859 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.480762    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 04:50:03.487327 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.487312    2565 manager.go:354] Recovery completed
Jul 16 04:50:03.503099 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.503079    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.510320 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510306    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.510392 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510336    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.510392 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510349    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.510949 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510935    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 04:50:03.510949 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510949    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 04:50:03.511061 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510960    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.511061 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.510965    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 04:50:03.512407 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.512338    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-249.ec2.internal.18c2abe54082efe8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-249.ec2.internal,UID:ip-10-0-138-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-249.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-249.ec2.internal,},FirstTimestamp:2026-07-16 04:50:03.510321128 +0000 UTC m=+0.430903916,LastTimestamp:2026-07-16 04:50:03.510321128 +0000 UTC m=+0.430903916,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-249.ec2.internal,}"
Jul 16 04:50:03.512515 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.512443    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.512515 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.512472    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.512515 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.512488    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.513983 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.513969    2565 policy_none.go:49] "None policy: Start"
Jul 16 04:50:03.514050 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.513998    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 04:50:03.514050 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.514033    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 04:50:03.517236 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.517221    2565 policy_none.go:47] "Start"
Jul 16 04:50:03.557416 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.557397    2565 manager.go:337] "Starting Device Plugin manager"
Jul 16 04:50:03.557500 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.557467    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 04:50:03.557500 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.557480    2565 server.go:87] "Starting device plugin registration server"
Jul 16 04:50:03.557726 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.557711    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 04:50:03.557787 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.557742    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 04:50:03.557950 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.557937    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 04:50:03.558032 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.558013    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 04:50:03.558032 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.558022    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 04:50:03.558846 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.558810    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 04:50:03.558918 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.558871    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:03.643048 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.642983    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 04:50:03.646969 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.644280    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 04:50:03.646969 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.644326    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 04:50:03.646969 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.644353    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 04:50:03.646969 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.644448    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 04:50:03.646969 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.646461    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:03.658552 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.658533    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.659383 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.659369    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.659449 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.659396    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.659449 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.659409    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.659449 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.659435    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.668481 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.668464    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.668568 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.668485    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-249.ec2.internal\": node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:03.693018 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.693001    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:03.744971 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.744947    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"]
Jul 16 04:50:03.745051 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.745021    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.746594 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.746580    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.746660 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.746607    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.746660 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.746618    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.748960 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.748947    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.749087 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749062    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.749131 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749103    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.749678 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749656    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.749773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749685    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.749773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749697    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.749773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749712    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.749773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749733    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.749773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.749748    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.751992 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.751974    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.752054 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.752010    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 04:50:03.752844 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.752814    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 04:50:03.752938 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.752857    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 04:50:03.752938 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.752871    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeHasSufficientPID"
Jul 16 04:50:03.775319 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.775292    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-249.ec2.internal\" not found" node="ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.776646 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.776627    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a56bf1f2a763dff711431b6502e7b62-config\") pod \"kube-apiserver-proxy-ip-10-0-138-249.ec2.internal\" (UID: \"3a56bf1f2a763dff711431b6502e7b62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.776741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.776649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0be9c52ecea3d08cb764c418f0a775e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal\" (UID: \"e0be9c52ecea3d08cb764c418f0a775e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.776741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.776665    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0be9c52ecea3d08cb764c418f0a775e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal\" (UID: \"e0be9c52ecea3d08cb764c418f0a775e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.779561 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.779540    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-249.ec2.internal\" not found" node="ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.793786 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.793767    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:03.876999 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.876974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a56bf1f2a763dff711431b6502e7b62-config\") pod \"kube-apiserver-proxy-ip-10-0-138-249.ec2.internal\" (UID: \"3a56bf1f2a763dff711431b6502e7b62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.877107 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.877022    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0be9c52ecea3d08cb764c418f0a775e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal\" (UID: \"e0be9c52ecea3d08cb764c418f0a775e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.877107 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.877041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0be9c52ecea3d08cb764c418f0a775e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal\" (UID: \"e0be9c52ecea3d08cb764c418f0a775e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.877107 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.877086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0be9c52ecea3d08cb764c418f0a775e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal\" (UID: \"e0be9c52ecea3d08cb764c418f0a775e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.877200 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.877086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a56bf1f2a763dff711431b6502e7b62-config\") pod \"kube-apiserver-proxy-ip-10-0-138-249.ec2.internal\" (UID: \"3a56bf1f2a763dff711431b6502e7b62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.877200 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:03.877086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0be9c52ecea3d08cb764c418f0a775e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal\" (UID: \"e0be9c52ecea3d08cb764c418f0a775e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:03.894667 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.894607    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:03.995380 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:03.995347    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:04.076703 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.076679    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:04.082675 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.082657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:04.095479 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.095451    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:04.196121 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.196035    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:04.296516 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.296492    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-249.ec2.internal\" not found"
Jul 16 04:50:04.319114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.319093    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:04.371301 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.371278    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:04.375043 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.375027    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:04.385108 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.385091    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 04:50:04.385959 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.385948    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal"
Jul 16 04:50:04.395563 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.395549    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 04:50:04.415681 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.415662    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 04:50:04.415814 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.415793    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 04:50:04.415900 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.415792    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 04:50:04.415900 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.415792    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 04:50:04.455991 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.455943    2565 apiserver.go:52] "Watching apiserver"
Jul 16 04:50:04.469985 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.469964    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 04:50:04.470317 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.470296    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-64wrz","openshift-network-diagnostics/network-check-target-rwzsw","kube-system/konnectivity-agent-4npp2","kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal","openshift-multus/multus-266pk","openshift-multus/multus-additional-cni-plugins-zk7dl","openshift-network-operator/iptables-alerter-f79z8","openshift-ovn-kubernetes/ovnkube-node-j28v9","kube-system/global-pull-secret-syncer-k44rb","openshift-cluster-node-tuning-operator/tuned-bkrmv","openshift-dns/node-resolver-7ttsm","openshift-image-registry/node-ca-k5s9w"]
Jul 16 04:50:04.473397 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.473380    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.475146 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.475129    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 04:50:04.475255 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.475185    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 04:50:04.475855 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.475815    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 04:50:04.475961 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.475898    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.476217 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.476199    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.476293 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.476240    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 04:50:04.476562 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.476547    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 04:50:04.476639 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.476546    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.476639 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.476554    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hmdbw\""
Jul 16 04:50:04.477875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.477859    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.477875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.477865    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.478007 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.477980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5fsvh\""
Jul 16 04:50:04.478404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478383    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.478616 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a9141ae-81d8-40dc-90b5-04a80209ebd4-hosts-file\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.478683 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478631    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9141ae-81d8-40dc-90b5-04a80209ebd4-tmp-dir\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.478683 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478664    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-kubelet\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478690    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-slash\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478720    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-run-netns\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478752    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-var-lib-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478775    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-log-socket\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478792    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwc6m\" (UniqueName: \"kubernetes.io/projected/c3067477-22a3-452f-b76a-0c7cc590818e-kube-api-access-wwc6m\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-systemd\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-etc-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.478892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478880    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-ovn\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-env-overrides\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3067477-22a3-452f-b76a-0c7cc590818e-ovn-node-metrics-cert\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478950    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-ovnkube-script-lib\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.478965    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj4xv\" (UniqueName: \"kubernetes.io/projected/3a9141ae-81d8-40dc-90b5-04a80209ebd4-kube-api-access-xj4xv\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479016    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-systemd-units\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-run-ovn-kubernetes\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479094 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479090    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-cni-bin\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-cni-netd\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479186    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479217    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-node-log\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.479328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.479239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-ovnkube-config\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.480565 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.480551    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 04:50:04.480669 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.480656    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 04:50:04.480741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.480729    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rtl9h\""
Jul 16 04:50:04.481082 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.481065    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.483494 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.483477    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5wlqb\""
Jul 16 04:50:04.483609 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.483493    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 04:50:04.483669 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.483633    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 04:45:04 +0000 UTC" deadline="2028-03-01 13:30:00.551820867 +0000 UTC"
Jul 16 04:50:04.483722 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.483672    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14264h39m56.068151104s"
Jul 16 04:50:04.483774 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.483719    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.483990 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.483978    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.484205 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.484194    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.486126 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.486108    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.486230 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.486132    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 04:50:04.486230 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.486218    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-875rr\""
Jul 16 04:50:04.486351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.486336    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.486639 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.486625    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 04:50:04.487386 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.487374    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.489283 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489263    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dbp4v\""
Jul 16 04:50:04.489374 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489319    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.489542 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489433    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 04:50:04.489777 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489763    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 04:50:04.489931 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489763    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 04:50:04.489931 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489794    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.489931 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.489812    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.491519 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.491495    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.491634 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.491613    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.491748 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.491734    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.491817 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.491803    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mknws\""
Jul 16 04:50:04.492074 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.492060    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 04:50:04.494057 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.493974    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:04.494057 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.494000    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wgql7\""
Jul 16 04:50:04.494057 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.494005    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 04:50:04.494695 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.494638    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:04.497272 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.497256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:04.497345 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.497312    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:04.499645 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.499627    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.499724 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.499683    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:04.501813 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.501794    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.503680 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.503661    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 04:50:04.503996 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.503982    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 04:50:04.504175 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.504161    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ghc4r\""
Jul 16 04:50:04.509254 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.509032    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jg482"
Jul 16 04:50:04.518497 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.518480    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jg482"
Jul 16 04:50:04.576030 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.576008    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 04:50:04.580136 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrkjs\" (UniqueName: \"kubernetes.io/projected/c295df82-b11d-407d-b0b8-9d94682e40ea-kube-api-access-qrkjs\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.580226 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580145    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-sys\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580226 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1ea6769-5131-421b-8c6e-bc860c512c25-host\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.580226 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580179    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-node-log\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580226 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580226    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-node-log\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580356 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580248    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-ovnkube-config\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580356 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysctl-d\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580356 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysctl-conf\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580356 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580296    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-run\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580356 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580309    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf819cf-b394-4051-a4a4-809f01569359-tmp\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580356 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-kubelet\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580368    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-var-lib-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/802cf865-e902-4310-8926-3943834ca81d-host-slash\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580414    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-var-lib-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580370    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-kubelet\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580437    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysconfig\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580459    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqb9v\" (UniqueName: \"kubernetes.io/projected/cae4612b-be73-479c-8902-109afde40010-kube-api-access-cqb9v\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580476    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-cnibin\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-etc-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-env-overrides\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580562    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-etc-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3067477-22a3-452f-b76a-0c7cc590818e-ovn-node-metrics-cert\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580670    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-ovnkube-config\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580698    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580788    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4dfaf89-1298-478a-95ac-7998743f8f51-kubelet-config\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4dfaf89-1298-478a-95ac-7998743f8f51-dbus\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5bf819cf-b394-4051-a4a4-809f01569359-etc-tuned\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-cni-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580938    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-systemd-units\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.580957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580951    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-env-overrides\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-systemd-units\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.580995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581020    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581032    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581044    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-kubernetes\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-hostroot\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581157    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-registration-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-openvswitch\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581178    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-os-release\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581198    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c295df82-b11d-407d-b0b8-9d94682e40ea-cni-binary-copy\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-slash\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581273    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-slash\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581301    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-run-netns\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.581401 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-run-netns\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-system-cni-dir\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-netns\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-ovn\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581456    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-ovnkube-script-lib\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581478    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-var-lib-kubelet\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581500    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-ovn\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581506    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-cni-bin\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-etc-kubernetes\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-cni-bin\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581617    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-cni-bin\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cml2q\" (UniqueName: \"kubernetes.io/projected/5bf819cf-b394-4051-a4a4-809f01569359-kube-api-access-cml2q\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1ea6769-5131-421b-8c6e-bc860c512c25-serviceca\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581681    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581698    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-cnibin\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581711    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-lib-modules\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a9141ae-81d8-40dc-90b5-04a80209ebd4-hosts-file\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.582089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9141ae-81d8-40dc-90b5-04a80209ebd4-tmp-dir\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581790    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a9141ae-81d8-40dc-90b5-04a80209ebd4-hosts-file\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/802cf865-e902-4310-8926-3943834ca81d-iptables-alerter-script\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581820    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz97z\" (UniqueName: \"kubernetes.io/projected/802cf865-e902-4310-8926-3943834ca81d-kube-api-access-zz97z\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-modprobe-d\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581908    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-device-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581931    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c19753c-e312-4509-b103-38488bfcbaa7-agent-certs\") pod \"konnectivity-agent-4npp2\" (UID: \"7c19753c-e312-4509-b103-38488bfcbaa7\") " pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c19753c-e312-4509-b103-38488bfcbaa7-konnectivity-ca\") pod \"konnectivity-agent-4npp2\" (UID: \"7c19753c-e312-4509-b103-38488bfcbaa7\") " pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.581981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwc6m\" (UniqueName: \"kubernetes.io/projected/c3067477-22a3-452f-b76a-0c7cc590818e-kube-api-access-wwc6m\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-sys-fs\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582025    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9141ae-81d8-40dc-90b5-04a80209ebd4-tmp-dir\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582101    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3067477-22a3-452f-b76a-0c7cc590818e-ovnkube-script-lib\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582180    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-kubelet\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582214    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-conf-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-daemon-config\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2q58\" (UniqueName: \"kubernetes.io/projected/a1ea6769-5131-421b-8c6e-bc860c512c25-kube-api-access-l2q58\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.582770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582284    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-etc-selinux\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-os-release\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-host\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582352    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-socket-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582385    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-run-ovn-kubernetes\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-cni-netd\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582441    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582470    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-run-ovn-kubernetes\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-socket-dir-parent\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-multus-certs\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-log-socket\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582562    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-system-cni-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582598    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-log-socket\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582540    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-host-cni-netd\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-k8s-cni-cncf-io\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582655    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-cni-multus\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.583304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582681    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-systemd\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582718    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj4xv\" (UniqueName: \"kubernetes.io/projected/3a9141ae-81d8-40dc-90b5-04a80209ebd4-kube-api-access-xj4xv\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwzkw\" (UniqueName: \"kubernetes.io/projected/2e6a8741-de9a-49f5-abfc-00122d8a77c7-kube-api-access-nwzkw\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582756    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3067477-22a3-452f-b76a-0c7cc590818e-run-systemd\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582772    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-systemd\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582800    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.583818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.582865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnttt\" (UniqueName: \"kubernetes.io/projected/e841a50b-ef95-406e-8481-d73401d1bd0e-kube-api-access-mnttt\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.584374 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.584357    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3067477-22a3-452f-b76a-0c7cc590818e-ovn-node-metrics-cert\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.592358 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.592336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwc6m\" (UniqueName: \"kubernetes.io/projected/c3067477-22a3-452f-b76a-0c7cc590818e-kube-api-access-wwc6m\") pod \"ovnkube-node-j28v9\" (UID: \"c3067477-22a3-452f-b76a-0c7cc590818e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.592581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.592564    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj4xv\" (UniqueName: \"kubernetes.io/projected/3a9141ae-81d8-40dc-90b5-04a80209ebd4-kube-api-access-xj4xv\") pod \"node-resolver-7ttsm\" (UID: \"3a9141ae-81d8-40dc-90b5-04a80209ebd4\") " pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.627274 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.627240    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0be9c52ecea3d08cb764c418f0a775e.slice/crio-ab7c552f9506113ded9f5f3ab30f2fa8807d2272f1a244b153a2ad7a661f777c WatchSource:0}: Error finding container ab7c552f9506113ded9f5f3ab30f2fa8807d2272f1a244b153a2ad7a661f777c: Status 404 returned error can't find the container with id ab7c552f9506113ded9f5f3ab30f2fa8807d2272f1a244b153a2ad7a661f777c
Jul 16 04:50:04.627702 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.627680    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a56bf1f2a763dff711431b6502e7b62.slice/crio-245b29dd6e0f84a9b7fa4e34a8e96dcbf7e0f5ab04789991d8956fddb03261ea WatchSource:0}: Error finding container 245b29dd6e0f84a9b7fa4e34a8e96dcbf7e0f5ab04789991d8956fddb03261ea: Status 404 returned error can't find the container with id 245b29dd6e0f84a9b7fa4e34a8e96dcbf7e0f5ab04789991d8956fddb03261ea
Jul 16 04:50:04.633132 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.633116    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 04:50:04.647326 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.647279    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal" event={"ID":"3a56bf1f2a763dff711431b6502e7b62","Type":"ContainerStarted","Data":"245b29dd6e0f84a9b7fa4e34a8e96dcbf7e0f5ab04789991d8956fddb03261ea"}
Jul 16 04:50:04.648307 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.648288    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal" event={"ID":"e0be9c52ecea3d08cb764c418f0a775e","Type":"ContainerStarted","Data":"ab7c552f9506113ded9f5f3ab30f2fa8807d2272f1a244b153a2ad7a661f777c"}
Jul 16 04:50:04.683921 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.683893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-lib-modules\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.684049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.683929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/802cf865-e902-4310-8926-3943834ca81d-iptables-alerter-script\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.684049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.683952    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz97z\" (UniqueName: \"kubernetes.io/projected/802cf865-e902-4310-8926-3943834ca81d-kube-api-access-zz97z\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.684049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.683972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-modprobe-d\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.684049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.683992    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-device-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.684049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684011    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c19753c-e312-4509-b103-38488bfcbaa7-agent-certs\") pod \"konnectivity-agent-4npp2\" (UID: \"7c19753c-e312-4509-b103-38488bfcbaa7\") " pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.684049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c19753c-e312-4509-b103-38488bfcbaa7-konnectivity-ca\") pod \"konnectivity-agent-4npp2\" (UID: \"7c19753c-e312-4509-b103-38488bfcbaa7\") " pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684055    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-sys-fs\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-kubelet\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684081    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-device-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684103    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-conf-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684079    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-lib-modules\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-daemon-config\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684134    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-kubelet\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684102    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-modprobe-d\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684149    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2q58\" (UniqueName: \"kubernetes.io/projected/a1ea6769-5131-421b-8c6e-bc860c512c25-kube-api-access-l2q58\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684159    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-sys-fs\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-etc-selinux\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-os-release\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-host\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684246    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-conf-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-socket-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684299    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:04.684325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-socket-dir-parent\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684355    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-socket-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-multus-certs\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684394    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-system-cni-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-etc-selinux\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-k8s-cni-cncf-io\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.684441    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-cni-multus\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-os-release\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684470    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-multus-certs\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwzkw\" (UniqueName: \"kubernetes.io/projected/2e6a8741-de9a-49f5-abfc-00122d8a77c7-kube-api-access-nwzkw\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.684508    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:05.184485802 +0000 UTC m=+2.105068588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-host\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-system-cni-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684524    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/802cf865-e902-4310-8926-3943834ca81d-iptables-alerter-script\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-socket-dir-parent\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684568    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-k8s-cni-cncf-io\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-cni-multus\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684597    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7c19753c-e312-4509-b103-38488bfcbaa7-konnectivity-ca\") pod \"konnectivity-agent-4npp2\" (UID: \"7c19753c-e312-4509-b103-38488bfcbaa7\") " pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684602    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-systemd\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684632    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-systemd\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684655    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684689    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnttt\" (UniqueName: \"kubernetes.io/projected/e841a50b-ef95-406e-8481-d73401d1bd0e-kube-api-access-mnttt\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684708    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrkjs\" (UniqueName: \"kubernetes.io/projected/c295df82-b11d-407d-b0b8-9d94682e40ea-kube-api-access-qrkjs\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684724    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-sys\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684745    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1ea6769-5131-421b-8c6e-bc860c512c25-host\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysctl-d\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684775    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysctl-conf\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-run\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684802    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf819cf-b394-4051-a4a4-809f01569359-tmp\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684803    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/802cf865-e902-4310-8926-3943834ca81d-host-slash\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.685909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysconfig\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqb9v\" (UniqueName: \"kubernetes.io/projected/cae4612b-be73-479c-8902-109afde40010-kube-api-access-cqb9v\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-cnibin\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysctl-conf\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684800    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-sys\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684976    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-cnibin\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684869    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1ea6769-5131-421b-8c6e-bc860c512c25-host\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-run\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685044    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysconfig\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685051    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-sysctl-d\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.684958    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/802cf865-e902-4310-8926-3943834ca81d-host-slash\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4dfaf89-1298-478a-95ac-7998743f8f51-kubelet-config\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4dfaf89-1298-478a-95ac-7998743f8f51-dbus\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685196    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5bf819cf-b394-4051-a4a4-809f01569359-etc-tuned\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685217    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-cni-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-kubernetes\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685263    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-hostroot\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685311    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-registration-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685323    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4dfaf89-1298-478a-95ac-7998743f8f51-dbus\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-os-release\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c295df82-b11d-407d-b0b8-9d94682e40ea-cni-binary-copy\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-os-release\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685436    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-system-cni-dir\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685463    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-netns\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685487    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-var-lib-kubelet\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-cni-bin\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685531    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-cni-binary-copy\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-etc-kubernetes\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685544    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cml2q\" (UniqueName: \"kubernetes.io/projected/5bf819cf-b394-4051-a4a4-809f01569359-kube-api-access-cml2q\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-daemon-config\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685582    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-hostroot\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.686967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685592    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1ea6769-5131-421b-8c6e-bc860c512c25-serviceca\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685627    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-etc-kubernetes\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685628    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e841a50b-ef95-406e-8481-d73401d1bd0e-system-cni-dir\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685672    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-cnibin\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685678    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cae4612b-be73-479c-8902-109afde40010-registration-dir\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685710    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-multus-cni-dir\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-etc-kubernetes\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.685744    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-cnibin\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685770    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-run-netns\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c295df82-b11d-407d-b0b8-9d94682e40ea-host-var-lib-cni-bin\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685821    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4dfaf89-1298-478a-95ac-7998743f8f51-kubelet-config\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.685853    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf819cf-b394-4051-a4a4-809f01569359-var-lib-kubelet\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.685859    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:05.185820568 +0000 UTC m=+2.106403345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.686137    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1ea6769-5131-421b-8c6e-bc860c512c25-serviceca\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.687581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.686154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c295df82-b11d-407d-b0b8-9d94682e40ea-cni-binary-copy\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.688093 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.686253    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e841a50b-ef95-406e-8481-d73401d1bd0e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.688093 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.687331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7c19753c-e312-4509-b103-38488bfcbaa7-agent-certs\") pod \"konnectivity-agent-4npp2\" (UID: \"7c19753c-e312-4509-b103-38488bfcbaa7\") " pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.688093 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.688086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf819cf-b394-4051-a4a4-809f01569359-tmp\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.688188 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.688112    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5bf819cf-b394-4051-a4a4-809f01569359-etc-tuned\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.692542 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.692524    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:04.692542 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.692544    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:04.692891 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.692555    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:04.692891 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:04.692607    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:50:05.192595258 +0000 UTC m=+2.113178048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:04.694541 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.694506    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2q58\" (UniqueName: \"kubernetes.io/projected/a1ea6769-5131-421b-8c6e-bc860c512c25-kube-api-access-l2q58\") pod \"node-ca-k5s9w\" (UID: \"a1ea6769-5131-421b-8c6e-bc860c512c25\") " pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.694714 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.694695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz97z\" (UniqueName: \"kubernetes.io/projected/802cf865-e902-4310-8926-3943834ca81d-kube-api-access-zz97z\") pod \"iptables-alerter-f79z8\" (UID: \"802cf865-e902-4310-8926-3943834ca81d\") " pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.695172 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.695154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cml2q\" (UniqueName: \"kubernetes.io/projected/5bf819cf-b394-4051-a4a4-809f01569359-kube-api-access-cml2q\") pod \"tuned-bkrmv\" (UID: \"5bf819cf-b394-4051-a4a4-809f01569359\") " pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.695737 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.695717    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrkjs\" (UniqueName: \"kubernetes.io/projected/c295df82-b11d-407d-b0b8-9d94682e40ea-kube-api-access-qrkjs\") pod \"multus-266pk\" (UID: \"c295df82-b11d-407d-b0b8-9d94682e40ea\") " pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.695805 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.695718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqb9v\" (UniqueName: \"kubernetes.io/projected/cae4612b-be73-479c-8902-109afde40010-kube-api-access-cqb9v\") pod \"aws-ebs-csi-driver-node-gfl57\" (UID: \"cae4612b-be73-479c-8902-109afde40010\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.696024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.696008    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwzkw\" (UniqueName: \"kubernetes.io/projected/2e6a8741-de9a-49f5-abfc-00122d8a77c7-kube-api-access-nwzkw\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:04.696084 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.696021    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnttt\" (UniqueName: \"kubernetes.io/projected/e841a50b-ef95-406e-8481-d73401d1bd0e-kube-api-access-mnttt\") pod \"multus-additional-cni-plugins-zk7dl\" (UID: \"e841a50b-ef95-406e-8481-d73401d1bd0e\") " pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.808578 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.808478    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:04.815666 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.815652    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7ttsm"
Jul 16 04:50:04.819514 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.819360    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3067477_22a3_452f_b76a_0c7cc590818e.slice/crio-1541cb7fb089901400c6243eed0fd58c04c6e0af4f03fd39bb7b23f2d458e500 WatchSource:0}: Error finding container 1541cb7fb089901400c6243eed0fd58c04c6e0af4f03fd39bb7b23f2d458e500: Status 404 returned error can't find the container with id 1541cb7fb089901400c6243eed0fd58c04c6e0af4f03fd39bb7b23f2d458e500
Jul 16 04:50:04.825286 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.825262    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9141ae_81d8_40dc_90b5_04a80209ebd4.slice/crio-bbd4b38f772f74c6a1333064cf4caa504c45b5665fc8350de34a897e31ba7dac WatchSource:0}: Error finding container bbd4b38f772f74c6a1333064cf4caa504c45b5665fc8350de34a897e31ba7dac: Status 404 returned error can't find the container with id bbd4b38f772f74c6a1333064cf4caa504c45b5665fc8350de34a897e31ba7dac
Jul 16 04:50:04.839557 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.839537    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:04.844138 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.844121    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k5s9w"
Jul 16 04:50:04.848608 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.848582    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c19753c_e312_4509_b103_38488bfcbaa7.slice/crio-e0ccaa1155318803b7f0a739d97febf19ae12385a1a73939f89c1b6f7fe816f2 WatchSource:0}: Error finding container e0ccaa1155318803b7f0a739d97febf19ae12385a1a73939f89c1b6f7fe816f2: Status 404 returned error can't find the container with id e0ccaa1155318803b7f0a739d97febf19ae12385a1a73939f89c1b6f7fe816f2
Jul 16 04:50:04.852874 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.852854    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ea6769_5131_421b_8c6e_bc860c512c25.slice/crio-2bbf37e1758c26e33bc29b7ae6216a582db6449e21ff5f00b295a3dab1548ee3 WatchSource:0}: Error finding container 2bbf37e1758c26e33bc29b7ae6216a582db6449e21ff5f00b295a3dab1548ee3: Status 404 returned error can't find the container with id 2bbf37e1758c26e33bc29b7ae6216a582db6449e21ff5f00b295a3dab1548ee3
Jul 16 04:50:04.867811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.867788    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57"
Jul 16 04:50:04.876995 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.876966    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae4612b_be73_479c_8902_109afde40010.slice/crio-de0123556a5060681f3dcc8f46fa16690a15d05d9042f8c7af0d04a1f6400e1f WatchSource:0}: Error finding container de0123556a5060681f3dcc8f46fa16690a15d05d9042f8c7af0d04a1f6400e1f: Status 404 returned error can't find the container with id de0123556a5060681f3dcc8f46fa16690a15d05d9042f8c7af0d04a1f6400e1f
Jul 16 04:50:04.883375 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.883355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zk7dl"
Jul 16 04:50:04.891248 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.891224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode841a50b_ef95_406e_8481_d73401d1bd0e.slice/crio-ef597371a177346f08fcf909d4c925f65a316cab8050f9554ab7fadcfdd5c5f6 WatchSource:0}: Error finding container ef597371a177346f08fcf909d4c925f65a316cab8050f9554ab7fadcfdd5c5f6: Status 404 returned error can't find the container with id ef597371a177346f08fcf909d4c925f65a316cab8050f9554ab7fadcfdd5c5f6
Jul 16 04:50:04.904371 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.904348    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f79z8"
Jul 16 04:50:04.912459 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.912434    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802cf865_e902_4310_8926_3943834ca81d.slice/crio-ee9a083a8323fe91e9b547c86f0349155f898f574a5f6f3470815c9b331d77b5 WatchSource:0}: Error finding container ee9a083a8323fe91e9b547c86f0349155f898f574a5f6f3470815c9b331d77b5: Status 404 returned error can't find the container with id ee9a083a8323fe91e9b547c86f0349155f898f574a5f6f3470815c9b331d77b5
Jul 16 04:50:04.916660 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.916643    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-266pk"
Jul 16 04:50:04.924324 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:04.924299    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkrmv"
Jul 16 04:50:04.926245 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.926224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc295df82_b11d_407d_b0b8_9d94682e40ea.slice/crio-1c93549e48f8b77028e71c214bb7a8fb51e1071a5eea62dfff57505169cfb9ea WatchSource:0}: Error finding container 1c93549e48f8b77028e71c214bb7a8fb51e1071a5eea62dfff57505169cfb9ea: Status 404 returned error can't find the container with id 1c93549e48f8b77028e71c214bb7a8fb51e1071a5eea62dfff57505169cfb9ea
Jul 16 04:50:04.932935 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:04.932907    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf819cf_b394_4051_a4a4_809f01569359.slice/crio-6948152dc3894b301fef7734ef9246bd0c71b21373ae1bcf891024284eb8b7fe WatchSource:0}: Error finding container 6948152dc3894b301fef7734ef9246bd0c71b21373ae1bcf891024284eb8b7fe: Status 404 returned error can't find the container with id 6948152dc3894b301fef7734ef9246bd0c71b21373ae1bcf891024284eb8b7fe
Jul 16 04:50:05.031402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.031375    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:05.189217 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.189179    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:05.189387 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.189308    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:05.189387 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.189382    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:06.189364781 +0000 UTC m=+3.109947558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:05.189536 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.189459    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:05.189536 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.189516    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:06.189499796 +0000 UTC m=+3.110082572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:05.189536 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.189313    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:05.290630 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.290589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:05.290872 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.290796    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:05.290985 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.290881    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:05.290985 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.290901    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:05.290985 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.290971    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:50:06.290951265 +0000 UTC m=+3.211534043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:05.328927 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.328874    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:05.522410 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.522228    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:45:05 +0000 UTC" deadline="2028-02-24 08:35:30.241705537 +0000 UTC"
Jul 16 04:50:05.522410 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.522296    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14115h45m24.719415359s"
Jul 16 04:50:05.647023 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.646994    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:05.647204 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.647118    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:05.647204 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.647201    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:05.647326 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:05.647288    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:05.667297 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.667210    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkrmv" event={"ID":"5bf819cf-b394-4051-a4a4-809f01569359","Type":"ContainerStarted","Data":"6948152dc3894b301fef7734ef9246bd0c71b21373ae1bcf891024284eb8b7fe"}
Jul 16 04:50:05.694500 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.694433    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-266pk" event={"ID":"c295df82-b11d-407d-b0b8-9d94682e40ea","Type":"ContainerStarted","Data":"1c93549e48f8b77028e71c214bb7a8fb51e1071a5eea62dfff57505169cfb9ea"}
Jul 16 04:50:05.709246 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.709202    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerStarted","Data":"ef597371a177346f08fcf909d4c925f65a316cab8050f9554ab7fadcfdd5c5f6"}
Jul 16 04:50:05.730453 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.730415    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57" event={"ID":"cae4612b-be73-479c-8902-109afde40010","Type":"ContainerStarted","Data":"de0123556a5060681f3dcc8f46fa16690a15d05d9042f8c7af0d04a1f6400e1f"}
Jul 16 04:50:05.740724 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.740690    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k5s9w" event={"ID":"a1ea6769-5131-421b-8c6e-bc860c512c25","Type":"ContainerStarted","Data":"2bbf37e1758c26e33bc29b7ae6216a582db6449e21ff5f00b295a3dab1548ee3"}
Jul 16 04:50:05.761771 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.761466    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4npp2" event={"ID":"7c19753c-e312-4509-b103-38488bfcbaa7","Type":"ContainerStarted","Data":"e0ccaa1155318803b7f0a739d97febf19ae12385a1a73939f89c1b6f7fe816f2"}
Jul 16 04:50:05.782418 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.782328    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"1541cb7fb089901400c6243eed0fd58c04c6e0af4f03fd39bb7b23f2d458e500"}
Jul 16 04:50:05.803341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.803298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f79z8" event={"ID":"802cf865-e902-4310-8926-3943834ca81d","Type":"ContainerStarted","Data":"ee9a083a8323fe91e9b547c86f0349155f898f574a5f6f3470815c9b331d77b5"}
Jul 16 04:50:05.826579 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:05.826501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7ttsm" event={"ID":"3a9141ae-81d8-40dc-90b5-04a80209ebd4","Type":"ContainerStarted","Data":"bbd4b38f772f74c6a1333064cf4caa504c45b5665fc8350de34a897e31ba7dac"}
Jul 16 04:50:06.198216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.198025    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:06.198216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.198085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:06.198216 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.198195    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:06.198496 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.198251    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:08.19823427 +0000 UTC m=+5.118817050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:06.198630 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.198580    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:06.198630 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.198622    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:08.198607746 +0000 UTC m=+5.119190520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:06.298920 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.298863    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:06.299161 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.299141    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:06.299240 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.299171    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:06.299240 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.299186    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:06.299344 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.299312    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:50:08.29929298 +0000 UTC m=+5.219875756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:06.523311 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.523100    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:45:05 +0000 UTC" deadline="2027-12-14 06:47:08.728778497 +0000 UTC"
Jul 16 04:50:06.523311 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.523166    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12385h57m2.205617278s"
Jul 16 04:50:06.595269 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.594803    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:06.645480 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:06.645449    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:06.645682 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:06.645600    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:07.278949 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:07.277060    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 04:50:07.647644 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:07.646886    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:07.647644 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:07.647029    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:07.647644 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:07.647420    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:07.647644 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:07.647514    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:08.216862 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:08.216106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:08.216862 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:08.216175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:08.216862 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.216341    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:08.216862 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.216401    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:12.216381817 +0000 UTC m=+9.136964596 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:08.216862 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.216782    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:08.216862 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.216834    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:12.216817266 +0000 UTC m=+9.137400043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:08.316614 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:08.316568    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:08.316787 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.316768    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:08.316842 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.316793    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:08.316842 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.316807    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:08.317004 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.316890    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:50:12.316854146 +0000 UTC m=+9.237436944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:08.645641 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:08.645069    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:08.645641 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:08.645227    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:09.646010 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:09.645973    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:09.646534 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:09.646115    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:09.646534 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:09.646174    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:09.646534 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:09.646269    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:10.645574 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:10.645532    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:10.645738 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:10.645692    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:11.646723 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:11.646031    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:11.646723 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:11.646137    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:11.647339 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:11.647313    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:11.647443 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:11.647418    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:12.252667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:12.252033    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:12.252667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:12.252086    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:12.252667 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.252196    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:12.252667 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.252255    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:20.252237396 +0000 UTC m=+17.172820176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:12.252667 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.252598    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:12.252667 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.252636    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:20.25262581 +0000 UTC m=+17.173208584 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:12.352367 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:12.352332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:12.352520 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.352496    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:12.352520 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.352512    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:12.352520 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.352522    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:12.352637 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.352564    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:50:20.352551109 +0000 UTC m=+17.273133882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:12.645143 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:12.645049    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:12.645317 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:12.645205    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:13.645945 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:13.645473    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:13.645945 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:13.645584    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:13.646438 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:13.645997    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:13.646438 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:13.646096    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:14.645164 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:14.644692    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:14.645164 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:14.644811    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:15.645325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:15.645289    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:15.645759 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:15.645294    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:15.645759 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:15.645417    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:15.645759 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:15.645489    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:16.644729 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:16.644701    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:16.644899 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:16.644803    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:17.645580 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:17.645506    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:17.645580 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:17.645547    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:17.646041 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:17.645643    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:17.646088 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:17.646036    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:18.645145 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:18.645111    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:18.645323 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:18.645236    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:19.644747 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:19.644713    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:19.645225 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:19.644852    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:19.645321 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:19.645300    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:19.645428 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:19.645407    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:20.316952 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:20.316915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:20.317181 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:20.316974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:20.317181 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.317081    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:20.317181 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.317083    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:20.317181 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.317127    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:36.317112924 +0000 UTC m=+33.237695697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:20.317181 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.317151    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:36.317132076 +0000 UTC m=+33.237714865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:20.418209 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:20.418171    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:20.418368 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.418305    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:20.418368 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.418323    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:20.418368 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.418332    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:20.418500 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.418376    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:50:36.418363188 +0000 UTC m=+33.338945960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:20.645099 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:20.645027    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:20.645473 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:20.645151    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:21.645042 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:21.644962    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:21.645204 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:21.644962    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:21.645204 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:21.645095    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:21.645204 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:21.645169    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:22.645318 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:22.645282    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:22.645736 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:22.645423    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:23.645621 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:23.645586    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:23.646090 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:23.645709    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:23.646090 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:23.645801    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:23.646090 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:23.645924    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:24.645227 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:24.645189    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:24.645415 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:24.645319    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:25.645445 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.645271    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:25.645965 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:25.645535    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:25.645965 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.645305    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:25.646060 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:25.645987    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:25.877680 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.877647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal" event={"ID":"3a56bf1f2a763dff711431b6502e7b62","Type":"ContainerStarted","Data":"c43a1c06b61c9704906e2a1ce0366e954d332581ed3a190652c1084611f8c3da"}
Jul 16 04:50:25.879161 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.879133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkrmv" event={"ID":"5bf819cf-b394-4051-a4a4-809f01569359","Type":"ContainerStarted","Data":"4102a645ee85d5925d672fb9291d0e8451c5fa9ebc6072fa7a64b32a633d9db0"}
Jul 16 04:50:25.880574 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.880512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-266pk" event={"ID":"c295df82-b11d-407d-b0b8-9d94682e40ea","Type":"ContainerStarted","Data":"3dfd4fdc5fcaf1f77b998883ae9f6c466405cd85f5ba15a243125fa5e58f5110"}
Jul 16 04:50:25.890837 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.890752    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-249.ec2.internal" podStartSLOduration=20.890739737 podStartE2EDuration="20.890739737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:25.890347037 +0000 UTC m=+22.810929830" watchObservedRunningTime="2026-07-16 04:50:25.890739737 +0000 UTC m=+22.811322531"
Jul 16 04:50:25.904423 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.904391    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bkrmv" podStartSLOduration=1.429907108 podStartE2EDuration="21.904378981s" totalImagesPullingTime="20.474471873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:25.904141492 +0000 UTC m=+22.824724285" watchObservedRunningTime="2026-07-16 04:50:25.904378981 +0000 UTC m=+22.824961773"
Jul 16 04:50:25.924179 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:25.924142    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-266pk" podStartSLOduration=1.410812566 podStartE2EDuration="21.924125885s" totalImagesPullingTime="20.513313319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:25.923990619 +0000 UTC m=+22.844573412" watchObservedRunningTime="2026-07-16 04:50:25.924125885 +0000 UTC m=+22.844708679"
Jul 16 04:50:26.645456 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.645304    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:26.645880 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:26.645532    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:26.883590 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.883564    2565 generic.go:356] "Generic (PLEG): container finished" podID="e841a50b-ef95-406e-8481-d73401d1bd0e" containerID="09768d534453a0cd2ea83a5bc5d1f4f837fac739a3094ed9cdc06019c54a1869" exitCode=0
Jul 16 04:50:26.883686 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.883635    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerDied","Data":"09768d534453a0cd2ea83a5bc5d1f4f837fac739a3094ed9cdc06019c54a1869"}
Jul 16 04:50:26.885441 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.885419    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57" event={"ID":"cae4612b-be73-479c-8902-109afde40010","Type":"ContainerStarted","Data":"cc488ce2a65da83a24a5e0ef6203756c7513597ec97138eaa0aa2845f66fc441"}
Jul 16 04:50:26.887136 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.887111    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k5s9w" event={"ID":"a1ea6769-5131-421b-8c6e-bc860c512c25","Type":"ContainerStarted","Data":"6178cc4e7d6978790616c8df4a5e240d26d944467b0ff72560dc7a7f56aa517f"}
Jul 16 04:50:26.888504 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.888480    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4npp2" event={"ID":"7c19753c-e312-4509-b103-38488bfcbaa7","Type":"ContainerStarted","Data":"dd430ac4554668f628951f299f181ce2d66261ee2e8b22e5beb675b032b6af0d"}
Jul 16 04:50:26.891526 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.891510    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:50:26.891870 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.891846    2565 generic.go:356] "Generic (PLEG): container finished" podID="c3067477-22a3-452f-b76a-0c7cc590818e" containerID="c1cb35eff99f901adc36d7b42a89e5d07a0678d97d0bde62683c537c652416a8" exitCode=1
Jul 16 04:50:26.891870 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.891860    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"09966b0437e045e631a7595dc0836c1916be2f13e6e84fa053767921ae8576a5"}
Jul 16 04:50:26.892014 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.891928    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"226da79ddc9405bbb1a0e1473795a69285372da155645ef82c3c9fa027a144b1"}
Jul 16 04:50:26.892014 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.891975    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerDied","Data":"c1cb35eff99f901adc36d7b42a89e5d07a0678d97d0bde62683c537c652416a8"}
Jul 16 04:50:26.892112 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.892015    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"26c321270530821cd8c5cb59760311c95b3d5ab2d43691ba33ea9ee4f1c54477"}
Jul 16 04:50:26.893439 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.893420    2565 generic.go:356] "Generic (PLEG): container finished" podID="e0be9c52ecea3d08cb764c418f0a775e" containerID="88ab6d4421374209ac2ccff8cae24f3d99493c5b917ef0398e7eb5be81b90ba9" exitCode=0
Jul 16 04:50:26.893522 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.893496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal" event={"ID":"e0be9c52ecea3d08cb764c418f0a775e","Type":"ContainerDied","Data":"88ab6d4421374209ac2ccff8cae24f3d99493c5b917ef0398e7eb5be81b90ba9"}
Jul 16 04:50:26.894935 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.894916    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f79z8" event={"ID":"802cf865-e902-4310-8926-3943834ca81d","Type":"ContainerStarted","Data":"b50275efb7a8298ef860861f5b9b2bf0548d2cff39c9e5e70ed4720907ec518c"}
Jul 16 04:50:26.896542 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.896518    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7ttsm" event={"ID":"3a9141ae-81d8-40dc-90b5-04a80209ebd4","Type":"ContainerStarted","Data":"4e001fdf91765f3a401ce42429a00894336c1ed3c79e3b3b72416060366d1b0f"}
Jul 16 04:50:26.917206 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.917171    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7ttsm" podStartSLOduration=2.337588933 podStartE2EDuration="22.917159359s" totalImagesPullingTime="20.579570426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:26.916698937 +0000 UTC m=+23.837281732" watchObservedRunningTime="2026-07-16 04:50:26.917159359 +0000 UTC m=+23.837742153"
Jul 16 04:50:26.930489 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.930456    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4npp2" podStartSLOduration=6.539502151 podStartE2EDuration="22.930445661s" totalImagesPullingTime="16.39094351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:26.930309641 +0000 UTC m=+23.850892436" watchObservedRunningTime="2026-07-16 04:50:26.930445661 +0000 UTC m=+23.851028482"
Jul 16 04:50:26.955609 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.955575    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f79z8" podStartSLOduration=2.494700314 podStartE2EDuration="22.955563959s" totalImagesPullingTime="20.460863645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:26.95547436 +0000 UTC m=+23.876057154" watchObservedRunningTime="2026-07-16 04:50:26.955563959 +0000 UTC m=+23.876146753"
Jul 16 04:50:26.955851 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:26.955814    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k5s9w" podStartSLOduration=2.435331345 podStartE2EDuration="22.955808472s" totalImagesPullingTime="20.520477127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:26.942635022 +0000 UTC m=+23.863217818" watchObservedRunningTime="2026-07-16 04:50:26.955808472 +0000 UTC m=+23.876391265"
Jul 16 04:50:27.644545 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.644520    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:27.644726 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:27.644625    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:27.644726 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.644703    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:27.644871 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:27.644798    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:27.771307 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.771159    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 04:50:27.900711 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.900626    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57" event={"ID":"cae4612b-be73-479c-8902-109afde40010","Type":"ContainerStarted","Data":"f11f2c511e308a7b398a4b63896a1799b554818ff3e3691f5bbbdd13e68e9375"}
Jul 16 04:50:27.903610 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.903593    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:50:27.904065 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.904031    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"21cbb039d5eb243adb2846ff243c3d6cc9899509ead492b6f7919c6edc479519"}
Jul 16 04:50:27.904136 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.904125    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"1881de8340165b918183dbf7b43fe8443f09512c6ea98d3080aa1fdae4a19cfe"}
Jul 16 04:50:27.965885 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.965857    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:27.966434 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:27.966414    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:28.571781 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.571672    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T04:50:27.771303731Z","UUID":"6dcd901d-6a6d-45ba-89c6-bebd16119330","Handler":null,"Name":"","Endpoint":""}
Jul 16 04:50:28.573283 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.573260    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 04:50:28.573409 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.573304    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 04:50:28.645025 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.644996    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:28.645197 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:28.645126    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:28.908575 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.908483    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal" event={"ID":"e0be9c52ecea3d08cb764c418f0a775e","Type":"ContainerStarted","Data":"1fcb897e7610fd075280d6b868d2ef6c234e892e39c8708321bc17f5fc953e7a"}
Jul 16 04:50:28.909227 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.908647    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:28.909227 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.909207    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4npp2"
Jul 16 04:50:28.923994 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:28.923945    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-249.ec2.internal" podStartSLOduration=23.923910459 podStartE2EDuration="23.923910459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:28.92349993 +0000 UTC m=+25.844082724" watchObservedRunningTime="2026-07-16 04:50:28.923910459 +0000 UTC m=+25.844493255"
Jul 16 04:50:29.644616 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:29.644545    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:29.644744 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:29.644649    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:29.644744 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:29.644703    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:29.644813 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:29.644789    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:29.913484 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:29.913402    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57" event={"ID":"cae4612b-be73-479c-8902-109afde40010","Type":"ContainerStarted","Data":"93c42603fc89071f5cdc7f9aa6708dd8c43ff8431a65504072649cde4640ea97"}
Jul 16 04:50:29.917180 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:29.917146    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:50:29.917534 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:29.917515    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"45e5ed1a7c5f4e40d14b6ba85835a3c3ca874087513d3b66e615f8e4851f200c"}
Jul 16 04:50:29.931753 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:29.931707    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfl57" podStartSLOduration=2.997266436 podStartE2EDuration="25.931689406s" totalImagesPullingTime="22.93442297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:29.931460087 +0000 UTC m=+26.852042881" watchObservedRunningTime="2026-07-16 04:50:29.931689406 +0000 UTC m=+26.852272200"
Jul 16 04:50:30.645152 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:30.645120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:30.645327 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:30.645237    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:31.645325 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.645174    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:31.645787 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.645256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:31.645787 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:31.645398    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:31.645787 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:31.645490    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:31.927524 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.927450    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:50:31.927886 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.927822    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"23c5529323b1808124117abe1780e88222e280bd02d18f6aa092f47be01b61c9"}
Jul 16 04:50:31.928095 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.928072    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:31.928175 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.928146    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:31.928226 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.928208    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:31.928277 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.928237    2565 scope.go:117] "RemoveContainer" containerID="c1cb35eff99f901adc36d7b42a89e5d07a0678d97d0bde62683c537c652416a8"
Jul 16 04:50:31.929792 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.929767    2565 generic.go:356] "Generic (PLEG): container finished" podID="e841a50b-ef95-406e-8481-d73401d1bd0e" containerID="0d13a8313f4ea3b173e72c17e32fecc88b93bdf05f829e8f8dc2e0befefb9f86" exitCode=0
Jul 16 04:50:31.929936 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.929801    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerDied","Data":"0d13a8313f4ea3b173e72c17e32fecc88b93bdf05f829e8f8dc2e0befefb9f86"}
Jul 16 04:50:31.945147 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.945127    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:31.945841 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:31.945810    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:50:32.645443 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:32.644999    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:32.645443 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:32.645120    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:32.935788 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:32.935604    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:50:32.936075 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:32.936056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" event={"ID":"c3067477-22a3-452f-b76a-0c7cc590818e","Type":"ContainerStarted","Data":"39514628ff0cc2dc77da7e19ba5760af608228350a2e385904dab11558c346c3"}
Jul 16 04:50:32.963137 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:32.963094    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9" podStartSLOduration=7.455901429 podStartE2EDuration="28.963080476s" totalImagesPullingTime="21.507179047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:32.962264491 +0000 UTC m=+29.882847284" watchObservedRunningTime="2026-07-16 04:50:32.963080476 +0000 UTC m=+29.883663269"
Jul 16 04:50:33.318774 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.318744    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rwzsw"]
Jul 16 04:50:33.318946 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.318899    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:33.318991 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:33.318978    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:33.319275 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.319255    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-64wrz"]
Jul 16 04:50:33.319351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.319342    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:33.319429 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:33.319414    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:33.332149 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.332127    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k44rb"]
Jul 16 04:50:33.332258 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.332218    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:33.332315 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:33.332291    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:33.406205 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.406179    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:33.485092 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.485013    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:33.939701 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.939624    2565 generic.go:356] "Generic (PLEG): container finished" podID="e841a50b-ef95-406e-8481-d73401d1bd0e" containerID="b44cc787b495751c6aa6e15e884864bf549ce86a2a2c0c6abd94c702b156ac9a" exitCode=0
Jul 16 04:50:33.940384 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:33.939709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerDied","Data":"b44cc787b495751c6aa6e15e884864bf549ce86a2a2c0c6abd94c702b156ac9a"}
Jul 16 04:50:34.645109 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:34.645080    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:34.645262 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:34.645170    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:34.645319 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:34.645260    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:34.645319 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:34.645172    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:34.645319 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:34.645175    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:34.645422 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:34.645340    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:35.946216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:35.946184    2565 generic.go:356] "Generic (PLEG): container finished" podID="e841a50b-ef95-406e-8481-d73401d1bd0e" containerID="f29795bac1504bd8b9f779f77c5b80e888815e0b7249acdc4b2dc94bd2a91c3f" exitCode=0
Jul 16 04:50:35.946673 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:35.946263    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerDied","Data":"f29795bac1504bd8b9f779f77c5b80e888815e0b7249acdc4b2dc94bd2a91c3f"}
Jul 16 04:50:36.341528 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:36.341293    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:36.341528 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.341482    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:36.341528 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:36.341520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:36.341769 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.341552    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret podName:c4dfaf89-1298-478a-95ac-7998743f8f51 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:08.341530113 +0000 UTC m=+65.262112902 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret") pod "global-pull-secret-syncer-k44rb" (UID: "c4dfaf89-1298-478a-95ac-7998743f8f51") : object "kube-system"/"original-pull-secret" not registered
Jul 16 04:50:36.341769 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.341592    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:36.341769 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.341637    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:08.341622781 +0000 UTC m=+65.262205553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 04:50:36.442078 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:36.442046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:36.442249 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.442197    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 04:50:36.442249 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.442220    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 04:50:36.442249 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.442232    2565 projected.go:196] Error preparing data for projected volume kube-api-access-xvxxk for pod openshift-network-diagnostics/network-check-target-rwzsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:36.442388 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.442291    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk podName:4a9d6215-2bf6-46e0-acbf-ca50c94a167b nodeName:}" failed. No retries permitted until 2026-07-16 04:51:08.442275361 +0000 UTC m=+65.362858153 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvxxk" (UniqueName: "kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk") pod "network-check-target-rwzsw" (UID: "4a9d6215-2bf6-46e0-acbf-ca50c94a167b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 04:50:36.645431 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:36.645356    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:36.645584 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:36.645464    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:36.645584 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.645475    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:36.645584 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:36.645484    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:36.645584 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.645532    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:36.645741 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:36.645602    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:37.008557 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:37.008414    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:38.420909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:38.420880    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:38.523342 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:38.523304    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:38.645252 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:38.645218    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:38.645252 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:38.645242    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:38.645483 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:38.645365    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:38.645483 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:38.645373    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k44rb" podUID="c4dfaf89-1298-478a-95ac-7998743f8f51"
Jul 16 04:50:38.645587 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:38.645493    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:50:38.645587 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:38.645556    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rwzsw" podUID="4a9d6215-2bf6-46e0-acbf-ca50c94a167b"
Jul 16 04:50:39.527818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.527744    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:39.584672 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.584648    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:39.626232 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.626208    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-249.ec2.internal" event="NodeReady"
Jul 16 04:50:39.626382 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.626330    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 04:50:39.658919 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.658892    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f9d757c49-d7pvp"]
Jul 16 04:50:39.664213 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.664189    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.667201 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.667170    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 04:50:39.667337 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.667206    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 04:50:39.667420 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.667398    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 04:50:39.667590 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.667464    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8whht\""
Jul 16 04:50:39.672310 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.672288    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-brtcs"]
Jul 16 04:50:39.675614 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.675560    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 04:50:39.676171 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.676149    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8g5qc"]
Jul 16 04:50:39.677032 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.677012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.679378 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.679359    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 04:50:39.679456 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.679384    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 04:50:39.679634 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.679619    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 04:50:39.679708 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.679671    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2p66f\""
Jul 16 04:50:39.680431 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.680413    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f9d757c49-d7pvp"]
Jul 16 04:50:39.680576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.680503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.682964 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.682946    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcm9d\""
Jul 16 04:50:39.683142 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.683124    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 04:50:39.683220 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.683174    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8g5qc"]
Jul 16 04:50:39.683329 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.683315    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 04:50:39.688884 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.688864    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-brtcs"]
Jul 16 04:50:39.769405 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769413    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6snct\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-kube-api-access-6snct\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8-cert\") pod \"ingress-canary-brtcs\" (UID: \"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8\") " pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.769560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769487    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-image-registry-private-configuration\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-bound-sa-token\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769766 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769584    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-registry-certificates\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769766 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769607    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n9cw\" (UniqueName: \"kubernetes.io/projected/12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8-kube-api-access-4n9cw\") pod \"ingress-canary-brtcs\" (UID: \"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8\") " pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.769766 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-trusted-ca\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769766 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6a33326-898b-4342-84c3-95e48c6036b9-ca-trust-extracted\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.769766 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.769757    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-installation-pull-secrets\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.870809 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870725    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6a33326-898b-4342-84c3-95e48c6036b9-ca-trust-extracted\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.870809 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-installation-pull-secrets\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6snct\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-kube-api-access-6snct\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8-cert\") pod \"ingress-canary-brtcs\" (UID: \"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8\") " pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-image-registry-private-configuration\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-bound-sa-token\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.870995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-registry-certificates\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871029 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871018    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n9cw\" (UniqueName: \"kubernetes.io/projected/12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8-kube-api-access-4n9cw\") pod \"ingress-canary-brtcs\" (UID: \"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8\") " pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871043    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871070    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-trusted-ca\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c73832e-cac6-4270-ae47-57021f21b236-config-volume\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871118    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c73832e-cac6-4270-ae47-57021f21b236-tmp-dir\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthsk\" (UniqueName: \"kubernetes.io/projected/4c73832e-cac6-4270-ae47-57021f21b236-kube-api-access-sthsk\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:39.871283    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:50:39.871333 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:39.871299    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:50:39.871596 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:39.871353    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:40.371335363 +0000 UTC m=+37.291918135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:50:39.871764 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.871737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6a33326-898b-4342-84c3-95e48c6036b9-ca-trust-extracted\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.872643 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.872618    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-trusted-ca\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.873264 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.873242    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-registry-certificates\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.876460 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.876441    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-image-registry-private-configuration\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.876543 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.876485    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-installation-pull-secrets\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.876543 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.876496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8-cert\") pod \"ingress-canary-brtcs\" (UID: \"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8\") " pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.880385 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.880335    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6snct\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-kube-api-access-6snct\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.880873 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.880848    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n9cw\" (UniqueName: \"kubernetes.io/projected/12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8-kube-api-access-4n9cw\") pod \"ingress-canary-brtcs\" (UID: \"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8\") " pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:39.881088 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.881072    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-bound-sa-token\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:39.971584 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.971562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c73832e-cac6-4270-ae47-57021f21b236-tmp-dir\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.971669 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.971597    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthsk\" (UniqueName: \"kubernetes.io/projected/4c73832e-cac6-4270-ae47-57021f21b236-kube-api-access-sthsk\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.971734 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.971699    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.971807 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:39.971793    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:50:39.971879 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.971846    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c73832e-cac6-4270-ae47-57021f21b236-tmp-dir\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.971879 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.971859    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c73832e-cac6-4270-ae47-57021f21b236-config-volume\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.971879 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:39.971869    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:40.47185025 +0000 UTC m=+37.392433022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:50:39.972334 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.972316    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c73832e-cac6-4270-ae47-57021f21b236-config-volume\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.979102 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.979081    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthsk\" (UniqueName: \"kubernetes.io/projected/4c73832e-cac6-4270-ae47-57021f21b236-kube-api-access-sthsk\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:39.987660 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:39.987642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-brtcs"
Jul 16 04:50:40.128714 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.128645    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-brtcs"]
Jul 16 04:50:40.132570 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:50:40.132415    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12db7a2f_2ccd_4bfb_bb1d_27ed64189eb8.slice/crio-906201f9f789b7deedab9dfbd60280c64bf852e57109bf351502df559e8a0e36 WatchSource:0}: Error finding container 906201f9f789b7deedab9dfbd60280c64bf852e57109bf351502df559e8a0e36: Status 404 returned error can't find the container with id 906201f9f789b7deedab9dfbd60280c64bf852e57109bf351502df559e8a0e36
Jul 16 04:50:40.375144 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.375115    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:40.375269 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:40.375229    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:50:40.375269 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:40.375240    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:50:40.375339 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:40.375287    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:41.375273639 +0000 UTC m=+38.295856411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:50:40.475937 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.475869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:40.476045 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:40.476025    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:50:40.476100 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:40.476089    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:41.476071687 +0000 UTC m=+38.396654460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:50:40.645075 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.645046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:50:40.645476 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.645046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:50:40.645476 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.645046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:50:40.647953 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.647652    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vpdjw\""
Jul 16 04:50:40.647953 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.647653    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 04:50:40.647953 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.647700    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 04:50:40.647953 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.647707    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 04:50:40.648192 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.648112    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njf2v\""
Jul 16 04:50:40.648418 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.648398    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 04:50:40.964114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.964076    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-brtcs" event={"ID":"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8","Type":"ContainerStarted","Data":"906201f9f789b7deedab9dfbd60280c64bf852e57109bf351502df559e8a0e36"}
Jul 16 04:50:40.967308 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.967272    2565 generic.go:356] "Generic (PLEG): container finished" podID="e841a50b-ef95-406e-8481-d73401d1bd0e" containerID="fbdc6d776b6bbd29db6198d1d02393572577b2981d59fed7df40296957b35f26" exitCode=0
Jul 16 04:50:40.967449 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:40.967337    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerDied","Data":"fbdc6d776b6bbd29db6198d1d02393572577b2981d59fed7df40296957b35f26"}
Jul 16 04:50:41.383678 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:41.383646    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:41.383855 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:41.383821    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:50:41.383903 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:41.383860    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:50:41.383967 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:41.383955    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:43.383934712 +0000 UTC m=+40.304517500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:50:41.484650 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:41.484616    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:41.484799 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:41.484735    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:50:41.484799 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:41.484785    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:43.484772353 +0000 UTC m=+40.405355125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:50:41.973772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:41.973741    2565 generic.go:356] "Generic (PLEG): container finished" podID="e841a50b-ef95-406e-8481-d73401d1bd0e" containerID="d29285277a308715f687fbc4061bb4f9a1b436756de34b5faba78d88da283e97" exitCode=0
Jul 16 04:50:41.973772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:41.973777    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerDied","Data":"d29285277a308715f687fbc4061bb4f9a1b436756de34b5faba78d88da283e97"}
Jul 16 04:50:42.979713 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:42.979633    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" event={"ID":"e841a50b-ef95-406e-8481-d73401d1bd0e","Type":"ContainerStarted","Data":"184f557ed093a11888c29c05358f11f70a81d1c4ba8aa2f9d509a910c30be704"}
Jul 16 04:50:42.980986 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:42.980961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-brtcs" event={"ID":"12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8","Type":"ContainerStarted","Data":"5ffdf11b3645d5346337d4044751290bcd62875d684fa50b5b43632d5c1086b1"}
Jul 16 04:50:43.000424 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.000382    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zk7dl" podStartSLOduration=7.928629524 podStartE2EDuration="39.000367188s" totalImagesPullingTime="31.071737664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:42.998860155 +0000 UTC m=+39.919442945" watchObservedRunningTime="2026-07-16 04:50:43.000367188 +0000 UTC m=+39.920949983"
Jul 16 04:50:43.014174 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.014122    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-brtcs" podStartSLOduration=1.485991899 podStartE2EDuration="4.014106264s" totalImagesPullingTime="2.528114365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:43.012285529 +0000 UTC m=+39.932868323" watchObservedRunningTime="2026-07-16 04:50:43.014106264 +0000 UTC m=+39.934689059"
Jul 16 04:50:43.124087 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.124062    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:43.177262 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.177231    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:43.337988 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.337915    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:43.391758 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.391726    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:43.397745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.397724    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:43.397861 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:43.397847    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:50:43.397908 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:43.397862    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:50:43.397937 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:43.397908    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:47.397895349 +0000 UTC m=+44.318478121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:50:43.498707 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:43.498675    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:43.498877 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:43.498790    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:50:43.498877 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:43.498851    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:47.498821164 +0000 UTC m=+44.419403936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:50:47.426190 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:47.426153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:47.426543 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:47.426262    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:50:47.426543 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:47.426273    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:50:47.426543 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:47.426327    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:55.426305688 +0000 UTC m=+52.346888460 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:50:47.526899 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:47.526869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:47.527048 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:47.527034    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:50:47.527100 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:47.527091    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:55.527074474 +0000 UTC m=+52.447657246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:50:48.442973 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:48.442935    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:48.489720 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:48.489697    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:49.849975 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:49.849939    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:49.898080 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:49.898048    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:55.480674 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:55.480640    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:50:55.481068 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:55.480815    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:50:55.481068 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:55.480855    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:50:55.481068 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:55.480921    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:11.480904384 +0000 UTC m=+68.401487160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:50:55.580972 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:55.580949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:50:55.581107 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:55.581082    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:50:55.581152 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:50:55.581130    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:11.581118086 +0000 UTC m=+68.501700859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:50:59.501555 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:59.501522    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:50:59.604138 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:50:59.604110    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:03.960252 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:03.960225    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j28v9"
Jul 16 04:51:04.107741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:04.107706    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:04.165812 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:04.165787    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:08.368221 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.368178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:51:08.368688 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.368262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:51:08.370721 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.370700    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 04:51:08.370803 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.370710    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 04:51:08.378511 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:08.378489    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 04:51:08.378610 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:08.378561    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:52:12.378540296 +0000 UTC m=+129.299123088 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : secret "metrics-daemon-secret" not found
Jul 16 04:51:08.381754 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.381734    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4dfaf89-1298-478a-95ac-7998743f8f51-original-pull-secret\") pod \"global-pull-secret-syncer-k44rb\" (UID: \"c4dfaf89-1298-478a-95ac-7998743f8f51\") " pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:51:08.469536 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.469501    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:51:08.471628 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.471609    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 04:51:08.482404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.482382    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 04:51:08.493452 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.493423    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvxxk\" (UniqueName: \"kubernetes.io/projected/4a9d6215-2bf6-46e0-acbf-ca50c94a167b-kube-api-access-xvxxk\") pod \"network-check-target-rwzsw\" (UID: \"4a9d6215-2bf6-46e0-acbf-ca50c94a167b\") " pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:51:08.557603 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.557578    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k44rb"
Jul 16 04:51:08.571889 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.571867    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njf2v\""
Jul 16 04:51:08.579508 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.579490    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:51:08.713708 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.713680    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k44rb"]
Jul 16 04:51:08.716588 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:51:08.716562    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4dfaf89_1298_478a_95ac_7998743f8f51.slice/crio-fb42b5bb5d5b03791c793d8bcbd30ca676b74f02f9d377e6ffe4583c2c85d31e WatchSource:0}: Error finding container fb42b5bb5d5b03791c793d8bcbd30ca676b74f02f9d377e6ffe4583c2c85d31e: Status 404 returned error can't find the container with id fb42b5bb5d5b03791c793d8bcbd30ca676b74f02f9d377e6ffe4583c2c85d31e
Jul 16 04:51:08.728672 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:08.728651    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rwzsw"]
Jul 16 04:51:08.731043 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:51:08.731020    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9d6215_2bf6_46e0_acbf_ca50c94a167b.slice/crio-48a23dd5d6d4d1296436a433a1bc66a07c8b61165ae2b93e8e749d608cf5c290 WatchSource:0}: Error finding container 48a23dd5d6d4d1296436a433a1bc66a07c8b61165ae2b93e8e749d608cf5c290: Status 404 returned error can't find the container with id 48a23dd5d6d4d1296436a433a1bc66a07c8b61165ae2b93e8e749d608cf5c290
Jul 16 04:51:09.042572 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:09.042478    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rwzsw" event={"ID":"4a9d6215-2bf6-46e0-acbf-ca50c94a167b","Type":"ContainerStarted","Data":"48a23dd5d6d4d1296436a433a1bc66a07c8b61165ae2b93e8e749d608cf5c290"}
Jul 16 04:51:09.043452 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:09.043429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k44rb" event={"ID":"c4dfaf89-1298-478a-95ac-7998743f8f51","Type":"ContainerStarted","Data":"fb42b5bb5d5b03791c793d8bcbd30ca676b74f02f9d377e6ffe4583c2c85d31e"}
Jul 16 04:51:10.386090 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:10.386038    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:10.439560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:10.439472    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:11.494740 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:11.494692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:51:11.495131 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:11.494869    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:51:11.495131 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:11.494892    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:51:11.495131 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:11.494953    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:43.494936972 +0000 UTC m=+100.415519745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:51:11.596156 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:11.596116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:51:11.596327 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:11.596267    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:51:11.596380 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:11.596330    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:43.596315516 +0000 UTC m=+100.516898291 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:51:13.058816 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:13.058776    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:13.110326 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:13.110290    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:14.056608 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:14.056508    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rwzsw" event={"ID":"4a9d6215-2bf6-46e0-acbf-ca50c94a167b","Type":"ContainerStarted","Data":"71474c5e7c75890a55eacf6e7514ac4077f0e90fb9598ebce3d702d037ae6dd3"}
Jul 16 04:51:14.056608 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:14.056572    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:51:14.057812 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:14.057789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k44rb" event={"ID":"c4dfaf89-1298-478a-95ac-7998743f8f51","Type":"ContainerStarted","Data":"3875b0ed1b31b98cdaf100f224f7af646be34e216f52397af95c5a66bea75543"}
Jul 16 04:51:14.071750 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:14.071710    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rwzsw" podStartSLOduration=65.18116711 podStartE2EDuration="1m10.071699155s" totalImagesPullingTime="4.890532045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:14.070273074 +0000 UTC m=+70.990855867" watchObservedRunningTime="2026-07-16 04:51:14.071699155 +0000 UTC m=+70.992281950"
Jul 16 04:51:14.083339 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:14.083294    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k44rb" podStartSLOduration=65.174796269 podStartE2EDuration="1m10.083277501s" totalImagesPullingTime="4.908481232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:14.082628868 +0000 UTC m=+71.003211661" watchObservedRunningTime="2026-07-16 04:51:14.083277501 +0000 UTC m=+71.003860296"
Jul 16 04:51:29.961340 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.961204    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"]
Jul 16 04:51:29.965545 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.965524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:29.967903 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.967878    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 16 04:51:29.968715 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.968697    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 16 04:51:29.968787 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.968716    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 16 04:51:29.968787 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.968744    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 16 04:51:29.974499 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:29.974474    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"]
Jul 16 04:51:30.019531 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.019502    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-klusterlet-config\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.019531 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.019536    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjktf\" (UniqueName: \"kubernetes.io/projected/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-kube-api-access-wjktf\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.019699 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.019579    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-tmp\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.120342 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.120310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-klusterlet-config\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.120342 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.120343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjktf\" (UniqueName: \"kubernetes.io/projected/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-kube-api-access-wjktf\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.120485 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.120377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-tmp\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.120774 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.120759    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-tmp\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.122768 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.122748    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-klusterlet-config\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.132396 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.132369    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjktf\" (UniqueName: \"kubernetes.io/projected/a16ad709-ccd5-466c-81dd-c8d9d0d176bd-kube-api-access-wjktf\") pod \"klusterlet-addon-workmgr-8655ddcd6c-ndfcp\" (UID: \"a16ad709-ccd5-466c-81dd-c8d9d0d176bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.278198 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.278138    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:30.393033 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:30.393004    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"]
Jul 16 04:51:30.395866 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:51:30.395815    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16ad709_ccd5_466c_81dd_c8d9d0d176bd.slice/crio-e5720a7de051394b14fbab4184ab44fea204f4818199177ca790ee34a8ff4f4d WatchSource:0}: Error finding container e5720a7de051394b14fbab4184ab44fea204f4818199177ca790ee34a8ff4f4d: Status 404 returned error can't find the container with id e5720a7de051394b14fbab4184ab44fea204f4818199177ca790ee34a8ff4f4d
Jul 16 04:51:31.105454 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:31.105379    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp" event={"ID":"a16ad709-ccd5-466c-81dd-c8d9d0d176bd","Type":"ContainerStarted","Data":"e5720a7de051394b14fbab4184ab44fea204f4818199177ca790ee34a8ff4f4d"}
Jul 16 04:51:34.189946 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:34.189916    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp" event={"ID":"a16ad709-ccd5-466c-81dd-c8d9d0d176bd","Type":"ContainerStarted","Data":"f1e17558edab660ab835fe368a7d583885ae742140873a1b605932d6af9f47f2"}
Jul 16 04:51:34.190307 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:34.190126    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:34.191405 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:34.191385    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:51:34.206108 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:34.206070    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp" podStartSLOduration=1.616118165 podStartE2EDuration="5.206057724s" totalImagesPullingTime="3.589939559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:34.205876809 +0000 UTC m=+91.126459603" watchObservedRunningTime="2026-07-16 04:51:34.206057724 +0000 UTC m=+91.126640567"
Jul 16 04:51:43.561162 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:43.561129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:51:43.561525 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:43.561271    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:51:43.561525 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:43.561288    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:51:43.561525 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:43.561343    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:52:47.561328115 +0000 UTC m=+164.481910887 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:51:43.661888 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:43.661864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:51:43.662019 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:43.661997    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:51:43.662082 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:51:43.662043    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:52:47.662030429 +0000 UTC m=+164.582613201 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:51:45.062957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:45.062930    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rwzsw"
Jul 16 04:51:48.454739 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:48.454704    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:51:48.507985 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:51:48.507962    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:52:12.456530 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:12.456493    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:52:12.456940 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:12.456627    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 04:52:12.456940 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:12.456697    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:14.456681519 +0000 UTC m=+251.377264291 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : secret "metrics-daemon-secret" not found
Jul 16 04:52:17.691106 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:17.691076    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:52:42.677755 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:42.677720    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp" podUID="c6a33326-898b-4342-84c3-95e48c6036b9"
Jul 16 04:52:42.693877 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:42.693852    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8g5qc" podUID="4c73832e-cac6-4270-ae47-57021f21b236"
Jul 16 04:52:43.349584 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:43.349560    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:52:43.349744 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:43.349560    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:52:43.663987 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:43.663910    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:52:47.582674 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:47.582634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") pod \"image-registry-5f9d757c49-d7pvp\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") " pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:52:47.583063 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:47.582753    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 04:52:47.583063 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:47.582764    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f9d757c49-d7pvp: secret "image-registry-tls" not found
Jul 16 04:52:47.583063 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:47.582809    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls podName:c6a33326-898b-4342-84c3-95e48c6036b9 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:49.582795497 +0000 UTC m=+286.503378270 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls") pod "image-registry-5f9d757c49-d7pvp" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9") : secret "image-registry-tls" not found
Jul 16 04:52:47.683663 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:47.683637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:52:47.683787 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:47.683729    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 04:52:47.683787 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:52:47.683776    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls podName:4c73832e-cac6-4270-ae47-57021f21b236 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:49.683764946 +0000 UTC m=+286.604347719 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls") pod "dns-default-8g5qc" (UID: "4c73832e-cac6-4270-ae47-57021f21b236") : secret "dns-default-metrics-tls" not found
Jul 16 04:52:58.644728 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:52:58.644647    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:53:34.190791 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:34.190754    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp" podUID="a16ad709-ccd5-466c-81dd-c8d9d0d176bd" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Jul 16 04:53:34.473158 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:34.473086    2565 generic.go:356] "Generic (PLEG): container finished" podID="a16ad709-ccd5-466c-81dd-c8d9d0d176bd" containerID="f1e17558edab660ab835fe368a7d583885ae742140873a1b605932d6af9f47f2" exitCode=1
Jul 16 04:53:34.473158 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:34.473144    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp" event={"ID":"a16ad709-ccd5-466c-81dd-c8d9d0d176bd","Type":"ContainerDied","Data":"f1e17558edab660ab835fe368a7d583885ae742140873a1b605932d6af9f47f2"}
Jul 16 04:53:34.473407 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:34.473395    2565 scope.go:117] "RemoveContainer" containerID="f1e17558edab660ab835fe368a7d583885ae742140873a1b605932d6af9f47f2"
Jul 16 04:53:35.477711 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:35.477679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp" event={"ID":"a16ad709-ccd5-466c-81dd-c8d9d0d176bd","Type":"ContainerStarted","Data":"d7a3e9411e35ef7f7128fdd8a9e10319e58bc72a8ddd86d509a8a2c43ed7dc73"}
Jul 16 04:53:35.478119 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:35.477934    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:53:35.478504 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:53:35.478486    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8655ddcd6c-ndfcp"
Jul 16 04:54:14.547488 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:14.547452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:54:14.547970 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:14.547600    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 04:54:14.547970 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:14.547662    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs podName:2e6a8741-de9a-49f5-abfc-00122d8a77c7 nodeName:}" failed. No retries permitted until 2026-07-16 04:56:16.54764701 +0000 UTC m=+373.468229785 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs") pod "network-metrics-daemon-64wrz" (UID: "2e6a8741-de9a-49f5-abfc-00122d8a77c7") : secret "metrics-daemon-secret" not found
Jul 16 04:54:18.267023 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.266983    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rt2bg"]
Jul 16 04:54:18.269985 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.269965    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.273707 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.273686    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qsn9b\""
Jul 16 04:54:18.282222 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.282204    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 04:54:18.282347 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.282221    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 04:54:18.282347 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.282223    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 16 04:54:18.282424 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.282224    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 16 04:54:18.291413 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.291399    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 16 04:54:18.302666 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.302646    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rt2bg"]
Jul 16 04:54:18.354607 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.354577    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"]
Jul 16 04:54:18.357327 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.357309    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"
Jul 16 04:54:18.359860 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.359817    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 16 04:54:18.360048 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.360032    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 16 04:54:18.360105 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.360074    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-s5npn\""
Jul 16 04:54:18.360397 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.360380    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-686d548b76-wnpsf"]
Jul 16 04:54:18.363303 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.363175    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.365502 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365483    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 04:54:18.365502 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365502    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 04:54:18.365644 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365515    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 04:54:18.365697 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365658    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 04:54:18.365753 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365725    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 04:54:18.365823 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365800    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"]
Jul 16 04:54:18.365959 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.365941    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8vbrn\""
Jul 16 04:54:18.366228 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.366210    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 04:54:18.371131 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.371112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24f9712e-b284-4b89-a8e7-3898c3ded11f-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.371225 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.371176    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24f9712e-b284-4b89-a8e7-3898c3ded11f-tmp\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.371225 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.371208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24f9712e-b284-4b89-a8e7-3898c3ded11f-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.371309 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.371275    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk299\" (UniqueName: \"kubernetes.io/projected/24f9712e-b284-4b89-a8e7-3898c3ded11f-kube-api-access-gk299\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.371309 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.371299    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/24f9712e-b284-4b89-a8e7-3898c3ded11f-snapshots\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.371384 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.371344    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24f9712e-b284-4b89-a8e7-3898c3ded11f-serving-cert\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.375573 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.375553    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-686d548b76-wnpsf"]
Jul 16 04:54:18.472117 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472096    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24f9712e-b284-4b89-a8e7-3898c3ded11f-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpcjn\" (UniqueName: \"kubernetes.io/projected/76da998e-d94d-4cb1-ae16-02ed647c23ac-kube-api-access-gpcjn\") pod \"volume-data-source-validator-78495597f6-6c9qm\" (UID: \"76da998e-d94d-4cb1-ae16-02ed647c23ac\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"
Jul 16 04:54:18.472260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472148    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/24f9712e-b284-4b89-a8e7-3898c3ded11f-snapshots\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczp8\" (UniqueName: \"kubernetes.io/projected/08124714-a993-46be-85ea-959d06600a08-kube-api-access-rczp8\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.472428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472291    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-default-certificate\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.472428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.472428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472336    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-stats-auth\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.472428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk299\" (UniqueName: \"kubernetes.io/projected/24f9712e-b284-4b89-a8e7-3898c3ded11f-kube-api-access-gk299\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472385    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24f9712e-b284-4b89-a8e7-3898c3ded11f-tmp\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24f9712e-b284-4b89-a8e7-3898c3ded11f-serving-cert\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472727 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.472727 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24f9712e-b284-4b89-a8e7-3898c3ded11f-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472727 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472708    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24f9712e-b284-4b89-a8e7-3898c3ded11f-tmp\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.472909 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/24f9712e-b284-4b89-a8e7-3898c3ded11f-snapshots\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.473013 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.472997    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24f9712e-b284-4b89-a8e7-3898c3ded11f-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.473098 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.473081    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24f9712e-b284-4b89-a8e7-3898c3ded11f-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.474762 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.474740    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24f9712e-b284-4b89-a8e7-3898c3ded11f-serving-cert\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.479635 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.479613    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk299\" (UniqueName: \"kubernetes.io/projected/24f9712e-b284-4b89-a8e7-3898c3ded11f-kube-api-access-gk299\") pod \"insights-operator-68d9dd94cf-rt2bg\" (UID: \"24f9712e-b284-4b89-a8e7-3898c3ded11f\") " pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.497964 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.497942    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:18.573713 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.573696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.573813 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.573738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpcjn\" (UniqueName: \"kubernetes.io/projected/76da998e-d94d-4cb1-ae16-02ed647c23ac-kube-api-access-gpcjn\") pod \"volume-data-source-validator-78495597f6-6c9qm\" (UID: \"76da998e-d94d-4cb1-ae16-02ed647c23ac\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"
Jul 16 04:54:18.573813 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.573770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczp8\" (UniqueName: \"kubernetes.io/projected/08124714-a993-46be-85ea-959d06600a08-kube-api-access-rczp8\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.573951 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:18.573852    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 04:54:18.573951 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.573878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-default-certificate\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.573951 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:18.573906    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:19.07389349 +0000 UTC m=+255.994476262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : secret "router-metrics-certs-default" not found
Jul 16 04:54:18.573951 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.573935    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.574145 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.573960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-stats-auth\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.574145 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:18.574048    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:19.074028551 +0000 UTC m=+255.994611350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : configmap references non-existent config key: service-ca.crt
Jul 16 04:54:18.576169 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.576140    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-default-certificate\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.576243 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.576182    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-stats-auth\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.578085 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.578068    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg"
Jul 16 04:54:18.582264 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.582095    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczp8\" (UniqueName: \"kubernetes.io/projected/08124714-a993-46be-85ea-959d06600a08-kube-api-access-rczp8\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:18.582264 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.582110    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpcjn\" (UniqueName: \"kubernetes.io/projected/76da998e-d94d-4cb1-ae16-02ed647c23ac-kube-api-access-gpcjn\") pod \"volume-data-source-validator-78495597f6-6c9qm\" (UID: \"76da998e-d94d-4cb1-ae16-02ed647c23ac\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"
Jul 16 04:54:18.586559 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.586537    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:18.667384 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.667099    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"
Jul 16 04:54:18.695906 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.695875    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-rt2bg"]
Jul 16 04:54:18.700008 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:18.699981    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f9712e_b284_4b89_a8e7_3898c3ded11f.slice/crio-ddf2de7e749e8347f4c230ca8664d7dfb64de5669284f575d813379c5feb6fbf WatchSource:0}: Error finding container ddf2de7e749e8347f4c230ca8664d7dfb64de5669284f575d813379c5feb6fbf: Status 404 returned error can't find the container with id ddf2de7e749e8347f4c230ca8664d7dfb64de5669284f575d813379c5feb6fbf
Jul 16 04:54:18.786814 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:18.786786    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm"]
Jul 16 04:54:18.789256 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:18.789233    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76da998e_d94d_4cb1_ae16_02ed647c23ac.slice/crio-66f9819d3330d405b140751052b652942df5a79ea2445deb96fd1db7cbf11459 WatchSource:0}: Error finding container 66f9819d3330d405b140751052b652942df5a79ea2445deb96fd1db7cbf11459: Status 404 returned error can't find the container with id 66f9819d3330d405b140751052b652942df5a79ea2445deb96fd1db7cbf11459
Jul 16 04:54:19.078028 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:19.077997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:19.078183 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:19.078071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:19.078183 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:19.078160    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 04:54:19.078255 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:19.078201    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:20.078179642 +0000 UTC m=+256.998762417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : configmap references non-existent config key: service-ca.crt
Jul 16 04:54:19.078255 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:19.078223    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:20.078215875 +0000 UTC m=+256.998798648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : secret "router-metrics-certs-default" not found
Jul 16 04:54:19.605283 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:19.605241    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg" event={"ID":"24f9712e-b284-4b89-a8e7-3898c3ded11f","Type":"ContainerStarted","Data":"ddf2de7e749e8347f4c230ca8664d7dfb64de5669284f575d813379c5feb6fbf"}
Jul 16 04:54:19.606621 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:19.606591    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm" event={"ID":"76da998e-d94d-4cb1-ae16-02ed647c23ac","Type":"ContainerStarted","Data":"66f9819d3330d405b140751052b652942df5a79ea2445deb96fd1db7cbf11459"}
Jul 16 04:54:20.087211 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:20.087175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:20.087392 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:20.087232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:20.087392 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:20.087349    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 04:54:20.087392 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:20.087359    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:22.087337563 +0000 UTC m=+259.007920352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : configmap references non-existent config key: service-ca.crt
Jul 16 04:54:20.087392 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:20.087384    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:22.087373953 +0000 UTC m=+259.007956731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : secret "router-metrics-certs-default" not found
Jul 16 04:54:20.611473 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:20.611434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm" event={"ID":"76da998e-d94d-4cb1-ae16-02ed647c23ac","Type":"ContainerStarted","Data":"621d0c2040a9abcb6b53a3282c1372f21d3441c9cf4b213869d4442ced3d129a"}
Jul 16 04:54:20.625629 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:20.625572    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6c9qm" podStartSLOduration=1.284749302 podStartE2EDuration="2.625554407s" totalImagesPullingTime="1.340805105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:20.62473693 +0000 UTC m=+257.545319727" watchObservedRunningTime="2026-07-16 04:54:20.625554407 +0000 UTC m=+257.546137203"
Jul 16 04:54:20.699292 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:20.699207    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:21.487006 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.486971    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8tss8"]
Jul 16 04:54:21.490230 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.490213    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.492358 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.492338    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q2w4b\""
Jul 16 04:54:21.492476 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.492459    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 04:54:21.492545 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.492532    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 04:54:21.500362 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.500345    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8tss8"]
Jul 16 04:54:21.508388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.508367    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:21.565590 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.565566    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:21.597744 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.597709    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6364c1b9-cca4-43fd-ba81-a829044ceabb-crio-socket\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.597908 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.597753    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kktcw\" (UniqueName: \"kubernetes.io/projected/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-api-access-kktcw\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.597908 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.597850    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6364c1b9-cca4-43fd-ba81-a829044ceabb-data-volume\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.597908 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.597886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.598013 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.597991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.615241 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.615213    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg" event={"ID":"24f9712e-b284-4b89-a8e7-3898c3ded11f","Type":"ContainerStarted","Data":"d2fc0f7f2a1ac4966adc415fd859768c935d7193e8fc1dae7728ce307a63259f"}
Jul 16 04:54:21.630951 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.630911    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg" podStartSLOduration=1.568637118 podStartE2EDuration="3.630898396s" totalImagesPullingTime="2.062261278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:21.63059258 +0000 UTC m=+258.551175374" watchObservedRunningTime="2026-07-16 04:54:21.630898396 +0000 UTC m=+258.551481190"
Jul 16 04:54:21.699178 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6364c1b9-cca4-43fd-ba81-a829044ceabb-crio-socket\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699322 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kktcw\" (UniqueName: \"kubernetes.io/projected/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-api-access-kktcw\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699322 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699250    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6364c1b9-cca4-43fd-ba81-a829044ceabb-crio-socket\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699322 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6364c1b9-cca4-43fd-ba81-a829044ceabb-data-volume\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699539 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699336    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699539 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699539 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:21.699482    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:21.699539 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:21.699535    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:22.199519031 +0000 UTC m=+259.120101803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:21.699737 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:21.699652    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 04:54:21.699737 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.699664    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6364c1b9-cca4-43fd-ba81-a829044ceabb-data-volume\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:21.699737 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:21.699708    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:22.199693836 +0000 UTC m=+259.120276628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : configmap "kube-rbac-proxy" not found
Jul 16 04:54:21.707174 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:21.707145    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kktcw\" (UniqueName: \"kubernetes.io/projected/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-api-access-kktcw\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:22.102358 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.102327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:22.102502 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.102393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:22.102502 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.102464    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 04:54:22.102502 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.102488    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:26.102475064 +0000 UTC m=+263.023057836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : configmap references non-existent config key: service-ca.crt
Jul 16 04:54:22.102603 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.102523    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:26.102510149 +0000 UTC m=+263.023092925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : secret "router-metrics-certs-default" not found
Jul 16 04:54:22.202710 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.202678    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:22.202872 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.202758    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:22.202872 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.202794    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 04:54:22.202944 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.202883    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:23.202861925 +0000 UTC m=+260.123444697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : configmap "kube-rbac-proxy" not found
Jul 16 04:54:22.202944 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.202884    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:22.202944 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:22.202925    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:23.202913144 +0000 UTC m=+260.123495916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:22.791003 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.790974    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:22.805424 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.805398    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"]
Jul 16 04:54:22.808380 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.808366    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"
Jul 16 04:54:22.810533 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.810515    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lm472\""
Jul 16 04:54:22.810725 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.810707    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 16 04:54:22.811430 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.811411    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 16 04:54:22.816263 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.816240    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"]
Jul 16 04:54:22.890912 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.890889    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:22.908039 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.908017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqj8n\" (UniqueName: \"kubernetes.io/projected/311fdd5e-de62-4222-a1cf-63f0690619d6-kube-api-access-mqj8n\") pod \"migrator-7f59b54f6c-9sfzx\" (UID: \"311fdd5e-de62-4222-a1cf-63f0690619d6\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"
Jul 16 04:54:22.961583 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:22.961562    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:23.009068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.009040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqj8n\" (UniqueName: \"kubernetes.io/projected/311fdd5e-de62-4222-a1cf-63f0690619d6-kube-api-access-mqj8n\") pod \"migrator-7f59b54f6c-9sfzx\" (UID: \"311fdd5e-de62-4222-a1cf-63f0690619d6\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"
Jul 16 04:54:23.016413 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.016393    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqj8n\" (UniqueName: \"kubernetes.io/projected/311fdd5e-de62-4222-a1cf-63f0690619d6-kube-api-access-mqj8n\") pod \"migrator-7f59b54f6c-9sfzx\" (UID: \"311fdd5e-de62-4222-a1cf-63f0690619d6\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"
Jul 16 04:54:23.117535 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.117512    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"
Jul 16 04:54:23.154130 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.154092    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:23.200352 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.200313    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:23.210501 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.210476    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:23.210621 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.210531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:23.210621 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:23.210604    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 04:54:23.210713 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:23.210623    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:23.210713 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:23.210670    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:25.210654251 +0000 UTC m=+262.131237035 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : configmap "kube-rbac-proxy" not found
Jul 16 04:54:23.210713 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:23.210685    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:25.210677779 +0000 UTC m=+262.131260551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:23.234813 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.234789    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx"]
Jul 16 04:54:23.237925 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:23.237897    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311fdd5e_de62_4222_a1cf_63f0690619d6.slice/crio-a54a1956eb1293c14ee808c6670cd26f4506cddad95b1e81dd490d4b62b2cc21 WatchSource:0}: Error finding container a54a1956eb1293c14ee808c6670cd26f4506cddad95b1e81dd490d4b62b2cc21: Status 404 returned error can't find the container with id a54a1956eb1293c14ee808c6670cd26f4506cddad95b1e81dd490d4b62b2cc21
Jul 16 04:54:23.261683 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.261667    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7ttsm_3a9141ae-81d8-40dc-90b5-04a80209ebd4/dns-node-resolver/0.log"
Jul 16 04:54:23.581871 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.581842    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:23.625462 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.625432    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx" event={"ID":"311fdd5e-de62-4222-a1cf-63f0690619d6","Type":"ContainerStarted","Data":"a54a1956eb1293c14ee808c6670cd26f4506cddad95b1e81dd490d4b62b2cc21"}
Jul 16 04:54:23.723250 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.723217    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:23.858744 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.858667    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:23.919437 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:23.919400    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:24.462174 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:24.462148    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k5s9w_a1ea6769-5131-421b-8c6e-bc860c512c25/node-ca/0.log"
Jul 16 04:54:24.630312 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:24.630240    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx" event={"ID":"311fdd5e-de62-4222-a1cf-63f0690619d6","Type":"ContainerStarted","Data":"629047e5413f478a223cdc0f9ca180bcfa092a03c61cfee48435c5863a96db5d"}
Jul 16 04:54:24.630312 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:24.630290    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx" event={"ID":"311fdd5e-de62-4222-a1cf-63f0690619d6","Type":"ContainerStarted","Data":"f7bb170917125cd34fbe0e51def29f2c09fd48bd33a132b9e46eae528d5ddd2a"}
Jul 16 04:54:24.646121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:24.646076    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9sfzx" podStartSLOduration=1.435278554 podStartE2EDuration="2.646062999s" totalImagesPullingTime="1.210784445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:24.644603967 +0000 UTC m=+261.565186762" watchObservedRunningTime="2026-07-16 04:54:24.646062999 +0000 UTC m=+261.566645858"
Jul 16 04:54:24.719433 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:24.719361    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:24.817545 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:24.817517    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:25.062041 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:25.061968    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brtcs_12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8/serve-healthcheck-canary/0.log"
Jul 16 04:54:25.228135 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:25.228102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:25.228255 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:25.228201    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:25.228255 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:25.228225    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 04:54:25.228341 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:25.228290    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:29.228265447 +0000 UTC m=+266.148848233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : configmap "kube-rbac-proxy" not found
Jul 16 04:54:25.228341 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:25.228321    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:25.228426 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:25.228355    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:29.228345074 +0000 UTC m=+266.148927849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:25.483975 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:25.483950    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:25.533835 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:25.533809    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:26.135413 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:26.135381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:26.135820 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:26.135438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:26.135820 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:26.135555    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 04:54:26.135820 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:26.135579    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:34.135554746 +0000 UTC m=+271.056137518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : configmap references non-existent config key: service-ca.crt
Jul 16 04:54:26.135820 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:26.135608    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs podName:08124714-a993-46be-85ea-959d06600a08 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:34.1355981 +0000 UTC m=+271.056180872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs") pod "router-default-686d548b76-wnpsf" (UID: "08124714-a993-46be-85ea-959d06600a08") : secret "router-metrics-certs-default" not found
Jul 16 04:54:26.288020 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:26.287996    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:26.720031 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:26.719987    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:26.880741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:26.880714    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:26.983104 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:26.983045    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:27.082304 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:27.082267    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:27.181392 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:27.181367    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:27.613800 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:27.613774    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:28.214039 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:28.214005    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:28.609752 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:28.609721    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:28.818494 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:28.818466    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:29.218223 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:29.218192    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:29.260671 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:29.260644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:29.260785 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:29.260692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:29.260785 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:29.260758    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 04:54:29.260906 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:29.260789    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:29.260906 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:29.260809    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:37.260795953 +0000 UTC m=+274.181378725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : configmap "kube-rbac-proxy" not found
Jul 16 04:54:29.260906 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:29.260850    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls podName:6364c1b9-cca4-43fd-ba81-a829044ceabb nodeName:}" failed. No retries permitted until 2026-07-16 04:54:37.260821699 +0000 UTC m=+274.181404472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8tss8" (UID: "6364c1b9-cca4-43fd-ba81-a829044ceabb") : secret "insights-runtime-extractor-tls" not found
Jul 16 04:54:29.373742 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:29.373698    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:29.634581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:29.634527    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:29.817096 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:29.817067    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:30.330940 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:30.330899    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:30.615637 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:30.615556    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:30.662779 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:30.662754    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:30.814969 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:30.814939    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:31.414612 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:31.414585    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:31.469942 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:31.469914    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:32.026820 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:32.026788    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:32.213460 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:32.213432    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:32.473482 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:32.473453    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:32.611181 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:32.611156    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:32.698464 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:32.698437    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:32.812058 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:32.811982    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:33.214529 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:33.214502    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:33.265871 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:33.265818    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:33.811381 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:33.811353    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:34.203376 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.203341    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:34.203523 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.203407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:34.204116 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.204087    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08124714-a993-46be-85ea-959d06600a08-service-ca-bundle\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:34.205930 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.205910    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08124714-a993-46be-85ea-959d06600a08-metrics-certs\") pod \"router-default-686d548b76-wnpsf\" (UID: \"08124714-a993-46be-85ea-959d06600a08\") " pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:34.213892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.213871    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:34.273482 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.273452    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:34.393843 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.393799    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-686d548b76-wnpsf"]
Jul 16 04:54:34.398297 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:34.398262    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08124714_a993_46be_85ea_959d06600a08.slice/crio-5061a3d2ff51297f5a5c430e8ebb092b5c82ef0a3b4347749e85675d5f7bd222 WatchSource:0}: Error finding container 5061a3d2ff51297f5a5c430e8ebb092b5c82ef0a3b4347749e85675d5f7bd222: Status 404 returned error can't find the container with id 5061a3d2ff51297f5a5c430e8ebb092b5c82ef0a3b4347749e85675d5f7bd222
Jul 16 04:54:34.412430 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.412402    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:34.657811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.657781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-686d548b76-wnpsf" event={"ID":"08124714-a993-46be-85ea-959d06600a08","Type":"ContainerStarted","Data":"709c3c748ea9ddde98f355d0755c1dc1879709167d0455297532620e1b936e6b"}
Jul 16 04:54:34.657811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.657821    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-686d548b76-wnpsf" event={"ID":"08124714-a993-46be-85ea-959d06600a08","Type":"ContainerStarted","Data":"5061a3d2ff51297f5a5c430e8ebb092b5c82ef0a3b4347749e85675d5f7bd222"}
Jul 16 04:54:34.674878 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.674814    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-686d548b76-wnpsf" podStartSLOduration=16.674796822 podStartE2EDuration="16.674796822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:34.673697071 +0000 UTC m=+271.594279855" watchObservedRunningTime="2026-07-16 04:54:34.674796822 +0000 UTC m=+271.595379615"
Jul 16 04:54:34.695008 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.694987    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:34.825294 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:34.825264    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:35.211754 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:35.211722    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:35.274629 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:35.274597    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:35.277185 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:35.277160    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:35.660479 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:35.660444    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:35.661678 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:35.661654    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-686d548b76-wnpsf"
Jul 16 04:54:35.811316 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:35.811272    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:36.212344 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:36.212316    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:36.415421 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:36.415393    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:37.025938 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.025901    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:37.127315 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.127280    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:37.229300 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.229259    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:37.326206 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.326181    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:37.332341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.332313    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:37.332442 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.332364    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:37.332951 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.332933    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6364c1b9-cca4-43fd-ba81-a829044ceabb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:37.334775 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.334752    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6364c1b9-cca4-43fd-ba81-a829044ceabb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8tss8\" (UID: \"6364c1b9-cca4-43fd-ba81-a829044ceabb\") " pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:37.399247 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.399207    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8tss8"
Jul 16 04:54:37.427696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.427666    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:37.526133 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.526096    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8tss8"]
Jul 16 04:54:37.529800 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:37.529769    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6364c1b9_cca4_43fd_ba81_a829044ceabb.slice/crio-31ff53ad40c986a96a97739961f0d7a003273403441d4a9bd7876ae3fc06f9f0 WatchSource:0}: Error finding container 31ff53ad40c986a96a97739961f0d7a003273403441d4a9bd7876ae3fc06f9f0: Status 404 returned error can't find the container with id 31ff53ad40c986a96a97739961f0d7a003273403441d4a9bd7876ae3fc06f9f0
Jul 16 04:54:37.667441 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.667336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8tss8" event={"ID":"6364c1b9-cca4-43fd-ba81-a829044ceabb","Type":"ContainerStarted","Data":"31ff53ad40c986a96a97739961f0d7a003273403441d4a9bd7876ae3fc06f9f0"}
Jul 16 04:54:37.812106 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:37.812073    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:42.685260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:42.685219    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8tss8" event={"ID":"6364c1b9-cca4-43fd-ba81-a829044ceabb","Type":"ContainerStarted","Data":"d7adbd8ea986cdb188023e362b1d8a08ca0501424c0b0b04e5966a5b439c1879"}
Jul 16 04:54:43.836074 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:43.836034    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:43.936946 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:43.936899    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.043936 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.043907    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.129153 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.129066    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.508892 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.508673    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f9d757c49-d7pvp"]
Jul 16 04:54:44.509024 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:44.508975    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp" podUID="c6a33326-898b-4342-84c3-95e48c6036b9"
Jul 16 04:54:44.540565 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.540529    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.593731 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.593693    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69cb7cb88c-xvzzp"]
Jul 16 04:54:44.608869 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.608840    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.612340 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.612315    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69cb7cb88c-xvzzp"]
Jul 16 04:54:44.633617 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.633585    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.691510 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.691482    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:54:44.694509 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694484    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d72c50d-2fe4-4b32-825b-d114314e1ce4-image-registry-private-configuration\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694636 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d72c50d-2fe4-4b32-825b-d114314e1ce4-trusted-ca\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694636 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694551    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d72c50d-2fe4-4b32-825b-d114314e1ce4-installation-pull-secrets\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694636 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d72c50d-2fe4-4b32-825b-d114314e1ce4-registry-certificates\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694785 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694668    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-bound-sa-token\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694785 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694714    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d72c50d-2fe4-4b32-825b-d114314e1ce4-ca-trust-extracted\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694894 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694792    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-registry-tls\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.694894 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.694811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzvzr\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-kube-api-access-dzvzr\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.697982 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.697949    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.701794 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.701773    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:54:44.795652 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795573    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6a33326-898b-4342-84c3-95e48c6036b9-ca-trust-extracted\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.795652 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795624    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-trusted-ca\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.795864 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795655    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-image-registry-private-configuration\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.795864 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795677    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-registry-certificates\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.795864 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795705    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-bound-sa-token\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.796008 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795875    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6a33326-898b-4342-84c3-95e48c6036b9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 04:54:44.796008 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.795955    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-installation-pull-secrets\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.796109 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796010    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6snct\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-kube-api-access-6snct\") pod \"c6a33326-898b-4342-84c3-95e48c6036b9\" (UID: \"c6a33326-898b-4342-84c3-95e48c6036b9\") "
Jul 16 04:54:44.796109 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796071    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:54:44.796197 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d72c50d-2fe4-4b32-825b-d114314e1ce4-image-registry-private-configuration\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.796197 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796143    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:54:44.796283 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796215    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d72c50d-2fe4-4b32-825b-d114314e1ce4-trusted-ca\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.796283 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796251    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d72c50d-2fe4-4b32-825b-d114314e1ce4-installation-pull-secrets\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.796380 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d72c50d-2fe4-4b32-825b-d114314e1ce4-registry-certificates\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.796380 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.796332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-bound-sa-token\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.797345 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797199    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d72c50d-2fe4-4b32-825b-d114314e1ce4-trusted-ca\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.797345 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d72c50d-2fe4-4b32-825b-d114314e1ce4-ca-trust-extracted\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.797345 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797305    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d72c50d-2fe4-4b32-825b-d114314e1ce4-registry-certificates\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.797576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-registry-tls\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.797576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzvzr\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-kube-api-access-dzvzr\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.797576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797519    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-trusted-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.797576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797536    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6a33326-898b-4342-84c3-95e48c6036b9-registry-certificates\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.797576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797552    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6a33326-898b-4342-84c3-95e48c6036b9-ca-trust-extracted\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.797863 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.797612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d72c50d-2fe4-4b32-825b-d114314e1ce4-ca-trust-extracted\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.799473 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.799412    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:54:44.799988 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.799876    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-kube-api-access-6snct" (OuterVolumeSpecName: "kube-api-access-6snct") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "kube-api-access-6snct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:54:44.800113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.800074    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:54:44.800113 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.800104    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d72c50d-2fe4-4b32-825b-d114314e1ce4-image-registry-private-configuration\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.800232 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.800106    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c6a33326-898b-4342-84c3-95e48c6036b9" (UID: "c6a33326-898b-4342-84c3-95e48c6036b9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:54:44.800626 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.800608    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-registry-tls\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.801358 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.801336    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d72c50d-2fe4-4b32-825b-d114314e1ce4-installation-pull-secrets\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.804188 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.804164    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-bound-sa-token\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.805018 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.804995    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzvzr\" (UniqueName: \"kubernetes.io/projected/0d72c50d-2fe4-4b32-825b-d114314e1ce4-kube-api-access-dzvzr\") pod \"image-registry-69cb7cb88c-xvzzp\" (UID: \"0d72c50d-2fe4-4b32-825b-d114314e1ce4\") " pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.898421 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.898392    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-image-registry-private-configuration\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.898421 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.898426    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-bound-sa-token\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.898910 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.898444    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6a33326-898b-4342-84c3-95e48c6036b9-installation-pull-secrets\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.898910 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.898456    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6snct\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-kube-api-access-6snct\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:44.922581 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.922554    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8whht\""
Jul 16 04:54:44.931364 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.931340    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:44.945016 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.944983    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:44.997237 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:44.997209    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:45.138187 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.138152    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:45.248474 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.248440    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:45.516155 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.516083    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:45.694588 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.694559    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9d757c49-d7pvp"
Jul 16 04:54:45.720103 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.720070    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:45.723603 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.723583    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f9d757c49-d7pvp"]
Jul 16 04:54:45.727178 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.727152    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f9d757c49-d7pvp"]
Jul 16 04:54:45.897894 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.897801    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69cb7cb88c-xvzzp"]
Jul 16 04:54:45.900399 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:45.900372    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d72c50d_2fe4_4b32_825b_d114314e1ce4.slice/crio-b878334b295f5896871c9a602bead0945bc3c431d81250685007dd81c5be94b4 WatchSource:0}: Error finding container b878334b295f5896871c9a602bead0945bc3c431d81250685007dd81c5be94b4: Status 404 returned error can't find the container with id b878334b295f5896871c9a602bead0945bc3c431d81250685007dd81c5be94b4
Jul 16 04:54:45.908600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:45.908582    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6a33326-898b-4342-84c3-95e48c6036b9-registry-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:54:46.007068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.007041    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:46.350597 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:46.350554    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8g5qc" podUID="4c73832e-cac6-4270-ae47-57021f21b236"
Jul 16 04:54:46.353404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.353366    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:46.700291 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.700208    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8tss8" event={"ID":"6364c1b9-cca4-43fd-ba81-a829044ceabb","Type":"ContainerStarted","Data":"d80c3e0ffedb7f7e4ffbe50fef961547bdb6f9a23eb40243f1944cf4dc967dc7"}
Jul 16 04:54:46.701819 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.701791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:54:46.701980 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.701822    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp" event={"ID":"0d72c50d-2fe4-4b32-825b-d114314e1ce4","Type":"ContainerStarted","Data":"99fe4f0b0bd6b73cef2e12cecf6d93a38641b973a04f6c4e2dccb30bc1b0fcb6"}
Jul 16 04:54:46.701980 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.701886    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp" event={"ID":"0d72c50d-2fe4-4b32-825b-d114314e1ce4","Type":"ContainerStarted","Data":"b878334b295f5896871c9a602bead0945bc3c431d81250685007dd81c5be94b4"}
Jul 16 04:54:46.701980 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.701925    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:54:46.720438 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.720397    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp" podStartSLOduration=2.720382282 podStartE2EDuration="2.720382282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:46.719433432 +0000 UTC m=+283.640016226" watchObservedRunningTime="2026-07-16 04:54:46.720382282 +0000 UTC m=+283.640965074"
Jul 16 04:54:46.838662 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.838588    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:46.938378 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.938328    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:46.993053 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:46.992963    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:47.648361 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:47.648329    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6a33326-898b-4342-84c3-95e48c6036b9" path="/var/lib/kubelet/pods/c6a33326-898b-4342-84c3-95e48c6036b9/volumes"
Jul 16 04:54:48.211386 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:48.211329    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:48.264221 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:48.264188    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:49.309967 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.309932    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:49.344198 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.344170    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-w4qpw"]
Jul 16 04:54:49.347280 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.347265    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.350989 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.350968    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 16 04:54:49.350989 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.350980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 16 04:54:49.351141 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.350995    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 04:54:49.351141 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.350980    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 04:54:49.351141 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.350980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ww9jk\""
Jul 16 04:54:49.351141 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.350974    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 04:54:49.356501 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.356481    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-w4qpw"]
Jul 16 04:54:49.375114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.375087    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:49.438607 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.438583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.438719 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.438626    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft8br\" (UniqueName: \"kubernetes.io/projected/6df33176-c55d-4e22-a193-638d28d77620-kube-api-access-ft8br\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.438719 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.438651    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.438823 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.438728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6df33176-c55d-4e22-a193-638d28d77620-metrics-client-ca\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.539433 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.539381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.539433 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.539421    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft8br\" (UniqueName: \"kubernetes.io/projected/6df33176-c55d-4e22-a193-638d28d77620-kube-api-access-ft8br\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.539576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.539456    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.539576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.539480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6df33176-c55d-4e22-a193-638d28d77620-metrics-client-ca\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.539677 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:49.539603    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 16 04:54:49.539677 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:49.539671    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls podName:6df33176-c55d-4e22-a193-638d28d77620 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:50.039650946 +0000 UTC m=+286.960233724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls") pod "prometheus-operator-64785b6675-w4qpw" (UID: "6df33176-c55d-4e22-a193-638d28d77620") : secret "prometheus-operator-tls" not found
Jul 16 04:54:49.540100 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.540084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6df33176-c55d-4e22-a193-638d28d77620-metrics-client-ca\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.541992 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.541973    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.547459 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.547424    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft8br\" (UniqueName: \"kubernetes.io/projected/6df33176-c55d-4e22-a193-638d28d77620-kube-api-access-ft8br\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:49.713781 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.713750    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8tss8" event={"ID":"6364c1b9-cca4-43fd-ba81-a829044ceabb","Type":"ContainerStarted","Data":"69b2a96d6a13b6ee217051d22aeaa0be7f24a6ad9d66215ed2661f0eb8774566"}
Jul 16 04:54:49.730013 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.729973    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8tss8" podStartSLOduration=17.185938252 podStartE2EDuration="28.729958606s" totalImagesPullingTime="11.544020354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:54:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:49.72931741 +0000 UTC m=+286.649900204" watchObservedRunningTime="2026-07-16 04:54:49.729958606 +0000 UTC m=+286.650541450"
Jul 16 04:54:49.741538 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.741515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:54:49.743841 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:49.743811    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c73832e-cac6-4270-ae47-57021f21b236-metrics-tls\") pod \"dns-default-8g5qc\" (UID: \"4c73832e-cac6-4270-ae47-57021f21b236\") " pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:54:50.004922 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:50.004898    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcm9d\""
Jul 16 04:54:50.013874 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:50.013856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:54:50.044114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:50.044089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:50.044239 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:50.044196    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 16 04:54:50.044277 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:50.044245    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls podName:6df33176-c55d-4e22-a193-638d28d77620 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:51.044231541 +0000 UTC m=+287.964814313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls") pod "prometheus-operator-64785b6675-w4qpw" (UID: "6df33176-c55d-4e22-a193-638d28d77620") : secret "prometheus-operator-tls" not found
Jul 16 04:54:50.135388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:50.135364    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8g5qc"]
Jul 16 04:54:50.137686 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:50.137659    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c73832e_cac6_4270_ae47_57021f21b236.slice/crio-cd56be0671c9e07e4122a3cc47393b29012fb929b1060b5928e57149d189ac91 WatchSource:0}: Error finding container cd56be0671c9e07e4122a3cc47393b29012fb929b1060b5928e57149d189ac91: Status 404 returned error can't find the container with id cd56be0671c9e07e4122a3cc47393b29012fb929b1060b5928e57149d189ac91
Jul 16 04:54:50.306862 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:50.306774    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:50.718254 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:50.718220    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8g5qc" event={"ID":"4c73832e-cac6-4270-ae47-57021f21b236","Type":"ContainerStarted","Data":"cd56be0671c9e07e4122a3cc47393b29012fb929b1060b5928e57149d189ac91"}
Jul 16 04:54:51.052558 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:51.052475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:51.054817 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:51.054789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6df33176-c55d-4e22-a193-638d28d77620-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w4qpw\" (UID: \"6df33176-c55d-4e22-a193-638d28d77620\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:51.156112 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:51.156082    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw"
Jul 16 04:54:51.301321 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:51.301290    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-w4qpw"]
Jul 16 04:54:51.304451 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:51.304383    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df33176_c55d_4e22_a193_638d28d77620.slice/crio-27e13fe8c16d0259495a113bc76dbca5e642e77beecb6089e44d86dd3ff671bb WatchSource:0}: Error finding container 27e13fe8c16d0259495a113bc76dbca5e642e77beecb6089e44d86dd3ff671bb: Status 404 returned error can't find the container with id 27e13fe8c16d0259495a113bc76dbca5e642e77beecb6089e44d86dd3ff671bb
Jul 16 04:54:51.723384 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:51.723347    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw" event={"ID":"6df33176-c55d-4e22-a193-638d28d77620","Type":"ContainerStarted","Data":"27e13fe8c16d0259495a113bc76dbca5e642e77beecb6089e44d86dd3ff671bb"}
Jul 16 04:54:52.727933 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:52.727875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8g5qc" event={"ID":"4c73832e-cac6-4270-ae47-57021f21b236","Type":"ContainerStarted","Data":"05a748f9cea8a171e4900a5e30080cc072f10bda05051b27437068519710d9da"}
Jul 16 04:54:52.728231 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:52.727960    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8g5qc" event={"ID":"4c73832e-cac6-4270-ae47-57021f21b236","Type":"ContainerStarted","Data":"58dbdc5eeca26417fa4a132f0aaa51652e5be6ebabcb9efcccf0e5b41220a273"}
Jul 16 04:54:52.728231 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:52.728047    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:54:52.744889 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:52.744852    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8g5qc" podStartSLOduration=252.088724035 podStartE2EDuration="4m13.744818292s" totalImagesPullingTime="1.656094257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:52.743737841 +0000 UTC m=+289.664320635" watchObservedRunningTime="2026-07-16 04:54:52.744818292 +0000 UTC m=+289.665401084"
Jul 16 04:54:52.953621 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:52.953553    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:53.050873 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:53.050814    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:53.732809 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:53.732780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw" event={"ID":"6df33176-c55d-4e22-a193-638d28d77620","Type":"ContainerStarted","Data":"5593ce6de00f90e89badaea570f875ac6c56d690a3f5a790f3f2d19eebfc27b8"}
Jul 16 04:54:53.733231 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:53.732841    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw" event={"ID":"6df33176-c55d-4e22-a193-638d28d77620","Type":"ContainerStarted","Data":"6eca7479d939204ec132b93671769d27293ae10ea6bafbf265b73bdd9a279608"}
Jul 16 04:54:53.749824 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:53.749773    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-w4qpw" podStartSLOduration=3.363843276 podStartE2EDuration="4.749759476s" totalImagesPullingTime="1.3859162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:53.748520631 +0000 UTC m=+290.669103427" watchObservedRunningTime="2026-07-16 04:54:53.749759476 +0000 UTC m=+290.670342269"
Jul 16 04:54:55.154302 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.154267    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.207763 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.207734    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.437418 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.437347    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.666141 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.666116    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.778663 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.778603    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.785232 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.785211    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s6xdj"]
Jul 16 04:54:55.787732 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.787718    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.789961 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.789944    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 04:54:55.790077 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.789977    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m8fw4\""
Jul 16 04:54:55.790077 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.789977    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 04:54:55.790077 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.789996    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 04:54:55.869423 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.869399    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.885535 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885514    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-sys\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885595 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885550    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885595 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5nsc\" (UniqueName: \"kubernetes.io/projected/321fede7-f4da-4ba9-bd18-fb521b7c82e1-kube-api-access-k5nsc\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885672 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-textfile\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885672 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885637    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-wtmp\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885672 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885700    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-tls\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885716    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-root\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.885770 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.885751    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/321fede7-f4da-4ba9-bd18-fb521b7c82e1-metrics-client-ca\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.970696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.970672    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:55.986796 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-textfile\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.986877 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-wtmp\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.986877 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.986946 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-tls\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.986946 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986902    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-root\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.986946 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986938    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/321fede7-f4da-4ba9-bd18-fb521b7c82e1-metrics-client-ca\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987065 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986960    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-wtmp\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987065 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.986988    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-sys\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987065 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987007    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-root\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987065 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987238 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nsc\" (UniqueName: \"kubernetes.io/projected/321fede7-f4da-4ba9-bd18-fb521b7c82e1-kube-api-access-k5nsc\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987238 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987082    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/321fede7-f4da-4ba9-bd18-fb521b7c82e1-sys\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987238 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:55.987132    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 16 04:54:55.987238 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987134    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-textfile\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987238 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:54:55.987191    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-tls podName:321fede7-f4da-4ba9-bd18-fb521b7c82e1 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:56.487173182 +0000 UTC m=+293.407755971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-tls") pod "node-exporter-s6xdj" (UID: "321fede7-f4da-4ba9-bd18-fb521b7c82e1") : secret "node-exporter-tls" not found
Jul 16 04:54:55.987453 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987432    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/321fede7-f4da-4ba9-bd18-fb521b7c82e1-metrics-client-ca\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.987453 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.987438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.989373 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.989357    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:55.994986 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:55.994966    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5nsc\" (UniqueName: \"kubernetes.io/projected/321fede7-f4da-4ba9-bd18-fb521b7c82e1-kube-api-access-k5nsc\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:56.089762 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.089738    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:56.189957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.189930    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:56.291328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.291307    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:56.490996 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.490926    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-tls\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:56.493232 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.493211    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/321fede7-f4da-4ba9-bd18-fb521b7c82e1-node-exporter-tls\") pod \"node-exporter-s6xdj\" (UID: \"321fede7-f4da-4ba9-bd18-fb521b7c82e1\") " pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:56.531424 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.531402    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:56.647638 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.647616    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:56.695957 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.695933    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s6xdj"
Jul 16 04:54:56.705018 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:54:56.704996    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321fede7_f4da_4ba9_bd18_fb521b7c82e1.slice/crio-4f3423dc92dfc2b2a3122d85853bd0443c0119e492422dfde01630909518c1cc WatchSource:0}: Error finding container 4f3423dc92dfc2b2a3122d85853bd0443c0119e492422dfde01630909518c1cc: Status 404 returned error can't find the container with id 4f3423dc92dfc2b2a3122d85853bd0443c0119e492422dfde01630909518c1cc
Jul 16 04:54:56.744469 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.744412    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6xdj" event={"ID":"321fede7-f4da-4ba9-bd18-fb521b7c82e1","Type":"ContainerStarted","Data":"4f3423dc92dfc2b2a3122d85853bd0443c0119e492422dfde01630909518c1cc"}
Jul 16 04:54:56.749116 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.749098    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:56.810927 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:56.810899    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:57.110926 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:57.110899    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:58.594427 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:58.594402    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:58.689362 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:58.689327    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:58.748586 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:58.748541    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:58.751751 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:58.751720    2565 generic.go:356] "Generic (PLEG): container finished" podID="321fede7-f4da-4ba9-bd18-fb521b7c82e1" containerID="7ba27ddf242a385262eb2d6ad307a847a5f3ccd80aaea6b56c292ad39c60caf2" exitCode=0
Jul 16 04:54:58.751877 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:58.751782    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6xdj" event={"ID":"321fede7-f4da-4ba9-bd18-fb521b7c82e1","Type":"ContainerDied","Data":"7ba27ddf242a385262eb2d6ad307a847a5f3ccd80aaea6b56c292ad39c60caf2"}
Jul 16 04:54:59.004149 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:59.004109    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:59.344074 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:59.344043    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:59.402384 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:59.402354    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:54:59.757402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:59.757327    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6xdj" event={"ID":"321fede7-f4da-4ba9-bd18-fb521b7c82e1","Type":"ContainerStarted","Data":"52552c755dcfc0110ce8ce94131ab85c605d2946bd22ca71b194d074739ab496"}
Jul 16 04:54:59.757402 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:59.757375    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6xdj" event={"ID":"321fede7-f4da-4ba9-bd18-fb521b7c82e1","Type":"ContainerStarted","Data":"d27ab24b44d6b2f1c8e3ea404df2c3255685c139e24a486d5e9f6ef39c860ffe"}
Jul 16 04:54:59.775689 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:54:59.775646    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s6xdj" podStartSLOduration=3.6812043389999998 podStartE2EDuration="4.775631919s" totalImagesPullingTime="1.024052982s" totalInitContainerRuntime="70.374598ms" isStatefulPod=true podCreationTimestamp="2026-07-16 04:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:59.774089929 +0000 UTC m=+296.694672745" watchObservedRunningTime="2026-07-16 04:54:59.775631919 +0000 UTC m=+296.696214713"
Jul 16 04:55:00.493012 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.492979    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:00.540292 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.540263    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8695d6cc6f-h5qdj"]
Jul 16 04:55:00.543456 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.543438    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.546587 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546570    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4bwwt\""
Jul 16 04:55:00.546708 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546594    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 04:55:00.546708 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546594    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 04:55:00.546708 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546577    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 04:55:00.546708 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546609    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 04:55:00.546937 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546846    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 04:55:00.546983 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.546939    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 04:55:00.547022 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.547011    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 04:55:00.551400 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.551380    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 04:55:00.552490 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.552472    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8695d6cc6f-h5qdj"]
Jul 16 04:55:00.624850 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.624813    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-oauth-config\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.624951 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.624859    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-trusted-ca-bundle\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.628852 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.625224    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5wqx\" (UniqueName: \"kubernetes.io/projected/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-kube-api-access-s5wqx\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.628852 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.625280    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-serving-cert\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.628852 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.625322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-oauth-serving-cert\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.628852 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.625360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-config\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.628852 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.625391    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-service-ca\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726379 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726354    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-oauth-serving-cert\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726450 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-config\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726450 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726398    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-service-ca\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726450 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726427    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-oauth-config\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726450 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-trusted-ca-bundle\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726626 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726587    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5wqx\" (UniqueName: \"kubernetes.io/projected/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-kube-api-access-s5wqx\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.726666 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.726648    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-serving-cert\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.727260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.727154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-oauth-serving-cert\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.727260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.727200    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-service-ca\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.727260 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.727223    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-config\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.727415 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.727350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-trusted-ca-bundle\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.728879 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.728857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-oauth-config\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.729160 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.729141    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-serving-cert\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.734293 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.734269    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5wqx\" (UniqueName: \"kubernetes.io/projected/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-kube-api-access-s5wqx\") pod \"console-8695d6cc6f-h5qdj\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") " pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.853691 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.853666    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:00.984131 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:00.984107    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8695d6cc6f-h5qdj"]
Jul 16 04:55:00.986520 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:55:00.986493    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55dc732_3e0c_497a_8dcc_4b0c7e21d9b3.slice/crio-d6d2c57e1894d0af7851cb0567062efd1dc17462890141c4f38969a9b6865e72 WatchSource:0}: Error finding container d6d2c57e1894d0af7851cb0567062efd1dc17462890141c4f38969a9b6865e72: Status 404 returned error can't find the container with id d6d2c57e1894d0af7851cb0567062efd1dc17462890141c4f38969a9b6865e72
Jul 16 04:55:01.645964 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:55:01.645917    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-64wrz" podUID="2e6a8741-de9a-49f5-abfc-00122d8a77c7"
Jul 16 04:55:01.767083 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.767045    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8695d6cc6f-h5qdj" event={"ID":"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3","Type":"ContainerStarted","Data":"d6d2c57e1894d0af7851cb0567062efd1dc17462890141c4f38969a9b6865e72"}
Jul 16 04:55:01.856793 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.856742    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:55:01.864213 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.864184    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.866747 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.866718    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 04:55:01.866873 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.866785    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 04:55:01.867071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867053    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l4h5v\""
Jul 16 04:55:01.867140 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867107    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 04:55:01.867535 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867265    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 04:55:01.867535 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867403    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 04:55:01.867535 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867436    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 04:55:01.867785 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867686    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 04:55:01.867970 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867928    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 04:55:01.867970 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.867969    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 04:55:01.868125 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.868003    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-87dqm17ldf01k\""
Jul 16 04:55:01.868181 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.868128    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 04:55:01.868245 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.868188    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 04:55:01.868245 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.868238    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 04:55:01.868343 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.868190    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 04:55:01.868792 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.868636    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 04:55:01.870081 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.870059    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 04:55:01.875513 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.875436    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:55:01.935257 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935174    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-config\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935272    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935337    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935364    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935389    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935413    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935435    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935509    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935547    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935600    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.935667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935654    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.936071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935704    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.936071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.936071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935769    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.936071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935862    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.936071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:01.936071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:01.935919    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjvql\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-kube-api-access-kjvql\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.036927 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.036822    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.036927 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.036894    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.036927 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.036923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjvql\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-kube-api-access-kjvql\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.037190 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.036951    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-config\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.037190 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.036982    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037293    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037418    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037468    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037624    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037767    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037787    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037793    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.038114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.037949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.039313 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.038993    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.040587 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.040563    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.041124 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.041097    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.041422 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.041399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043064 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.042015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043064 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.042410    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043064 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.042586    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043064 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.043027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043545 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.043404    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043545 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.043510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.043902 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.043878    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.044082 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.044059    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.044363 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.044340    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.046043 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.046001    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjvql\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-kube-api-access-kjvql\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.046647 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.046603    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.046732 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.046703    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.047645 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.047604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-config\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.048213 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.048166    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.054334 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.054310    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.178453 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.178413    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:02.342277 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.342144    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:55:02.344674 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:55:02.344645    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9725736f_7fb3_4667_91e4_9bb30fce52fe.slice/crio-025ae3c4729ca55833bf4e0d5270878a4d372da9ec0880f0bd1c50f06affd179 WatchSource:0}: Error finding container 025ae3c4729ca55833bf4e0d5270878a4d372da9ec0880f0bd1c50f06affd179: Status 404 returned error can't find the container with id 025ae3c4729ca55833bf4e0d5270878a4d372da9ec0880f0bd1c50f06affd179
Jul 16 04:55:02.735155 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.735125    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8g5qc"
Jul 16 04:55:02.772110 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:02.772055    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"025ae3c4729ca55833bf4e0d5270878a4d372da9ec0880f0bd1c50f06affd179"}
Jul 16 04:55:03.020406 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:03.020321    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:04.171722 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.171701    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:55:04.174196 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.174086    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 04:55:04.174944 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.174923    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 04:55:04.781379 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.781345    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4" exitCode=0
Jul 16 04:55:04.781555 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.781424    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4"}
Jul 16 04:55:04.782652 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.782636    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 04:55:04.783170 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.783122    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8695d6cc6f-h5qdj" event={"ID":"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3","Type":"ContainerStarted","Data":"9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e"}
Jul 16 04:55:04.821255 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.821214    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8695d6cc6f-h5qdj" podStartSLOduration=1.660868113 podStartE2EDuration="4.821201556s" totalImagesPullingTime="3.160333443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:55:04.820735315 +0000 UTC m=+301.741318135" watchObservedRunningTime="2026-07-16 04:55:04.821201556 +0000 UTC m=+301.741784349"
Jul 16 04:55:04.936290 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.936265    2565 patch_prober.go:28] interesting pod/image-registry-69cb7cb88c-xvzzp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 16 04:55:04.936419 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:04.936307    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp" podUID="0d72c50d-2fe4-4b32-825b-d114314e1ce4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 04:55:05.151541 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:05.151507    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:05.249538 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:05.249500    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:06.322591 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:06.322546    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:06.422420 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:06.422387    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:07.710634 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:07.710610    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69cb7cb88c-xvzzp"
Jul 16 04:55:07.829018 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:07.828977    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:07.885389 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:07.885343    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:08.126244 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:08.126169    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:08.173080 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:08.173048    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:08.323516 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:08.323480    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:08.421399 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:08.421378    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:08.801620 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:08.801530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea"}
Jul 16 04:55:08.802021 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:08.801624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315"}
Jul 16 04:55:09.145643 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:09.145610    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:09.247458 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:09.247422    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:09.300314 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:09.300253    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:10.811739 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.811705    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31"}
Jul 16 04:55:10.812171 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.811791    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27"}
Jul 16 04:55:10.812171 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.811819    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f"}
Jul 16 04:55:10.812171 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.811872    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerStarted","Data":"586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c"}
Jul 16 04:55:10.838797 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.838746    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.860231849 podStartE2EDuration="9.838730093s" totalImagesPullingTime="6.960835023s" totalInitContainerRuntime="17.663221ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:55:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:55:10.837083753 +0000 UTC m=+307.757666547" watchObservedRunningTime="2026-07-16 04:55:10.838730093 +0000 UTC m=+307.759312891"
Jul 16 04:55:10.854163 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.854140    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:10.854286 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.854202    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:10.858857 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:10.858817    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:11.818622 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:11.818596    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:12.178877 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:12.178788    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:55:14.147068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:14.147032    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:14.197817 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:14.197787    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:16.645016 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:16.644979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:55:17.159138 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:17.159108    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:17.258991 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:17.258960    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:20.176721 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:20.176685    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:20.229912 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:20.229890    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:20.754128 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:20.754096    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:20.812666 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:20.812637    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:25.298642 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:25.298613    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:25.399346 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:25.399314    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:29.248600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:29.248567    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:29.349647 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:29.349600    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:30.198110 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:30.198078    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8695d6cc6f-h5qdj"]
Jul 16 04:55:30.314433 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:30.314398    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:30.413037 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:30.413007    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:30.466997 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:30.466914    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:46.838886 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:46.838821    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:46.890132 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:46.890103    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:55:51.952353 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:51.952327    2565 generic.go:356] "Generic (PLEG): container finished" podID="24f9712e-b284-4b89-a8e7-3898c3ded11f" containerID="d2fc0f7f2a1ac4966adc415fd859768c935d7193e8fc1dae7728ce307a63259f" exitCode=0
Jul 16 04:55:51.952693 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:51.952361    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg" event={"ID":"24f9712e-b284-4b89-a8e7-3898c3ded11f","Type":"ContainerDied","Data":"d2fc0f7f2a1ac4966adc415fd859768c935d7193e8fc1dae7728ce307a63259f"}
Jul 16 04:55:51.952693 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:51.952653    2565 scope.go:117] "RemoveContainer" containerID="d2fc0f7f2a1ac4966adc415fd859768c935d7193e8fc1dae7728ce307a63259f"
Jul 16 04:55:52.957341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:52.957308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-rt2bg" event={"ID":"24f9712e-b284-4b89-a8e7-3898c3ded11f","Type":"ContainerStarted","Data":"f84c68416f8945df0304f362e878aa35d0ec40259b21b17a59b214871b666422"}
Jul 16 04:55:55.226431 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.226359    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8695d6cc6f-h5qdj" podUID="c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" containerName="console" containerID="cri-o://9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e" gracePeriod=15
Jul 16 04:55:55.469461 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.469431    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:55.565281 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565208    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-serving-cert\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565281 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565247    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-trusted-ca-bundle\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565281 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565267    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-service-ca\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565518 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565298    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-oauth-config\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565518 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565408    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-oauth-serving-cert\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565518 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565474    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-config\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565518 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565503    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5wqx\" (UniqueName: \"kubernetes.io/projected/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-kube-api-access-s5wqx\") pod \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\" (UID: \"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3\") "
Jul 16 04:55:55.565714 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565684    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:55:55.565874 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565823    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:55:55.566159 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.565891    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-config" (OuterVolumeSpecName: "console-config") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:55:55.566159 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.566006    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:55:55.567576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.567553    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:55:55.567689 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.567606    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:55:55.567689 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.567646    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-kube-api-access-s5wqx" (OuterVolumeSpecName: "kube-api-access-s5wqx") pod "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" (UID: "c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3"). InnerVolumeSpecName "kube-api-access-s5wqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:55:55.666666 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666645    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-serving-cert\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.666773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666731    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-trusted-ca-bundle\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.666773 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666757    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.666875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666772    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-oauth-config\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.666875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666787    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-oauth-serving-cert\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.666875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666800    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-console-config\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.666875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.666821    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5wqx\" (UniqueName: \"kubernetes.io/projected/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3-kube-api-access-s5wqx\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:55:55.967895 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.967865    2565 generic.go:356] "Generic (PLEG): container finished" podID="c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" containerID="9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e" exitCode=0
Jul 16 04:55:55.967895 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.967898    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8695d6cc6f-h5qdj" event={"ID":"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3","Type":"ContainerDied","Data":"9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e"}
Jul 16 04:55:55.968128 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.967917    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8695d6cc6f-h5qdj"
Jul 16 04:55:55.968128 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.967932    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8695d6cc6f-h5qdj" event={"ID":"c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3","Type":"ContainerDied","Data":"d6d2c57e1894d0af7851cb0567062efd1dc17462890141c4f38969a9b6865e72"}
Jul 16 04:55:55.968128 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.967952    2565 scope.go:117] "RemoveContainer" containerID="9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e"
Jul 16 04:55:55.976480 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.976465    2565 scope.go:117] "RemoveContainer" containerID="9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e"
Jul 16 04:55:55.976730 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:55:55.976698    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e\": container with ID starting with 9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e not found: ID does not exist" containerID="9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e"
Jul 16 04:55:55.976810 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.976738    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e"} err="failed to get container status \"9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e\": rpc error: code = NotFound desc = could not find container \"9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e\": container with ID starting with 9632984f2f190d32e19b5891ce1e48b2f4d834b3371c5855053147e0f94e517e not found: ID does not exist"
Jul 16 04:55:55.983956 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.983932    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8695d6cc6f-h5qdj"]
Jul 16 04:55:55.987370 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:55.987348    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8695d6cc6f-h5qdj"]
Jul 16 04:55:57.649501 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:55:57.649467    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" path="/var/lib/kubelet/pods/c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3/volumes"
Jul 16 04:56:02.178771 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:02.178739    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:02.194114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:02.194089    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:03.008541 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:03.008512    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:11.786669 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:11.786638    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:11.843331 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:11.843303    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:12.682091 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:12.682060    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:12.782583 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:12.782557    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:15.632693 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:15.632658    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:15.684562 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:15.684528    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:15.762840 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:15.762810    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:15.829038 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:15.829004    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:16.368636 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.368604    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:16.425757 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.425719    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:16.634635 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.634548    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:56:16.637011 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.636989    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e6a8741-de9a-49f5-abfc-00122d8a77c7-metrics-certs\") pod \"network-metrics-daemon-64wrz\" (UID: \"2e6a8741-de9a-49f5-abfc-00122d8a77c7\") " pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:56:16.648417 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.648394    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vpdjw\""
Jul 16 04:56:16.657073 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.657054    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-64wrz"
Jul 16 04:56:16.799642 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:16.799605    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-64wrz"]
Jul 16 04:56:16.802425 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:56:16.802397    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6a8741_de9a_49f5_abfc_00122d8a77c7.slice/crio-d0c795eafa2d9f32fd46ed0d948d9763d9070c8dbaaeade2fa1c22eb6d5f19f0 WatchSource:0}: Error finding container d0c795eafa2d9f32fd46ed0d948d9763d9070c8dbaaeade2fa1c22eb6d5f19f0: Status 404 returned error can't find the container with id d0c795eafa2d9f32fd46ed0d948d9763d9070c8dbaaeade2fa1c22eb6d5f19f0
Jul 16 04:56:17.044238 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:17.044154    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-64wrz" event={"ID":"2e6a8741-de9a-49f5-abfc-00122d8a77c7","Type":"ContainerStarted","Data":"d0c795eafa2d9f32fd46ed0d948d9763d9070c8dbaaeade2fa1c22eb6d5f19f0"}
Jul 16 04:56:18.049270 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:18.049177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-64wrz" event={"ID":"2e6a8741-de9a-49f5-abfc-00122d8a77c7","Type":"ContainerStarted","Data":"3f256deb0fa5499c16e587963ede4923e7e6df801b83c0489e046c735f20eb01"}
Jul 16 04:56:18.049270 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:18.049238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-64wrz" event={"ID":"2e6a8741-de9a-49f5-abfc-00122d8a77c7","Type":"ContainerStarted","Data":"4ee22a60de5000689439cc8621d8c555a00415ba5b2a147ff1d328a347a942c1"}
Jul 16 04:56:18.065900 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:18.065848    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-64wrz" podStartSLOduration=373.259702209 podStartE2EDuration="6m14.06581142s" totalImagesPullingTime="806.109211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:56:18.064477899 +0000 UTC m=+374.985060692" watchObservedRunningTime="2026-07-16 04:56:18.06581142 +0000 UTC m=+374.986394213"
Jul 16 04:56:18.146328 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:18.146296    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:18.246936 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:18.246907    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:20.264399 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.264369    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:56:20.264901 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.264683    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="prometheus" containerID="cri-o://571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315" gracePeriod=600
Jul 16 04:56:20.264901 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.264872    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-web" containerID="cri-o://6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f" gracePeriod=600
Jul 16 04:56:20.265040 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.264922    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-thanos" containerID="cri-o://30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31" gracePeriod=600
Jul 16 04:56:20.265040 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.264872    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy" containerID="cri-o://08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27" gracePeriod=600
Jul 16 04:56:20.265040 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.265012    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="thanos-sidecar" containerID="cri-o://586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c" gracePeriod=600
Jul 16 04:56:20.265040 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:20.265031    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="config-reloader" containerID="cri-o://21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea" gracePeriod=600
Jul 16 04:56:21.069526 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069492    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31" exitCode=0
Jul 16 04:56:21.069526 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069521    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27" exitCode=0
Jul 16 04:56:21.069526 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069529    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c" exitCode=0
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069537    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea" exitCode=0
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069544    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315" exitCode=0
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069562    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31"}
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27"}
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c"}
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea"}
Jul 16 04:56:21.069811 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.069760    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315"}
Jul 16 04:56:21.514390 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.514367    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:21.578848 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.578739    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-metrics-client-certs\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.578848 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.578786    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-0\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.578848 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.578844    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-kubelet-serving-ca-bundle\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579093 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.578871    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-kube-rbac-proxy\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579093 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579022    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-web-config\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579093 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579067    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-config-out\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579241 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-db\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579241 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579144    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579241 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579173    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjvql\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-kube-api-access-kjvql\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579241 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579210    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579241 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579241    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-1\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579289    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-thanos-prometheus-http-client-file\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579318    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-tls-assets\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579339    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-grpc-tls\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579372    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-tls\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579398    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-2\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579397    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579426    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-config\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.579471 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579454    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-serving-certs-ca-bundle\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.580068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579486    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-metrics-client-ca\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.580068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579512    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-trusted-ca-bundle\") pod \"9725736f-7fb3-4667-91e4-9bb30fce52fe\" (UID: \"9725736f-7fb3-4667-91e4-9bb30fce52fe\") "
Jul 16 04:56:21.580068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.579757    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.580068 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.580007    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.580255 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.580135    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.580741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.580460    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.580741 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.580679    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.581156 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.581127    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.581436 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.581412    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:56:21.582147 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.582120    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 04:56:21.582407 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.582381    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.582981 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.582940    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.584086 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.584063    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-config" (OuterVolumeSpecName: "config") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.585218 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.585169    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.585688 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.585600    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.585688 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.585638    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-config-out" (OuterVolumeSpecName: "config-out") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 04:56:21.585866 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.585683    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:56:21.585930 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.585873    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.586704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.586675    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.586798 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.586705    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.587397 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.587283    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-kube-api-access-kjvql" (OuterVolumeSpecName: "kube-api-access-kjvql") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "kube-api-access-kjvql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:56:21.594667 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.594633    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-web-config" (OuterVolumeSpecName: "web-config") pod "9725736f-7fb3-4667-91e4-9bb30fce52fe" (UID: "9725736f-7fb3-4667-91e4-9bb30fce52fe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:56:21.680370 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680348    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-metrics-client-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680370 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680369    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680379    2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-metrics-client-certs\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680388    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680397    2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-kube-rbac-proxy\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680406    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-web-config\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680414    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-config-out\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680421    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-db\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680430    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680438    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjvql\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-kube-api-access-kjvql\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680445    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680453    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680461    2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680469    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9725736f-7fb3-4667-91e4-9bb30fce52fe-tls-assets\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680476    2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-grpc-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680478 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680483    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680864 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680491    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680864 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680499    2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9725736f-7fb3-4667-91e4-9bb30fce52fe-config\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:21.680864 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:21.680506    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9725736f-7fb3-4667-91e4-9bb30fce52fe-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:56:22.076000 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.075969    2565 generic.go:356] "Generic (PLEG): container finished" podID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerID="6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f" exitCode=0
Jul 16 04:56:22.076134 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.076041    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.076134 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.076043    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f"}
Jul 16 04:56:22.076134 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.076109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9725736f-7fb3-4667-91e4-9bb30fce52fe","Type":"ContainerDied","Data":"025ae3c4729ca55833bf4e0d5270878a4d372da9ec0880f0bd1c50f06affd179"}
Jul 16 04:56:22.076267 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.076143    2565 scope.go:117] "RemoveContainer" containerID="30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31"
Jul 16 04:56:22.083346 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.083306    2565 scope.go:117] "RemoveContainer" containerID="08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27"
Jul 16 04:56:22.089780 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.089760    2565 scope.go:117] "RemoveContainer" containerID="6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f"
Jul 16 04:56:22.094587 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.094567    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:56:22.096737 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.096722    2565 scope.go:117] "RemoveContainer" containerID="586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c"
Jul 16 04:56:22.098928 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.098804    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:56:22.103374 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.103352    2565 scope.go:117] "RemoveContainer" containerID="21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea"
Jul 16 04:56:22.109481 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.109467    2565 scope.go:117] "RemoveContainer" containerID="571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315"
Jul 16 04:56:22.116473 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.116458    2565 scope.go:117] "RemoveContainer" containerID="cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4"
Jul 16 04:56:22.122989 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.122975    2565 scope.go:117] "RemoveContainer" containerID="30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31"
Jul 16 04:56:22.123215 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123198    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:56:22.123273 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.123225    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31\": container with ID starting with 30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31 not found: ID does not exist" containerID="30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31"
Jul 16 04:56:22.123273 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123244    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31"} err="failed to get container status \"30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31\": rpc error: code = NotFound desc = could not find container \"30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31\": container with ID starting with 30dc33e6d089bf67e8369ce19f88d4166493e1a7b8dea9a284ebcb33986cba31 not found: ID does not exist"
Jul 16 04:56:22.123273 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123260    2565 scope.go:117] "RemoveContainer" containerID="08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27"
Jul 16 04:56:22.123447 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123434    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="thanos-sidecar"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123448    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="thanos-sidecar"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123458    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="config-reloader"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123465    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="config-reloader"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123473    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="init-config-reloader"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123482    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="init-config-reloader"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123489    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy"
Jul 16 04:56:22.123491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123494    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.123491    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27\": container with ID starting with 08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27 not found: ID does not exist" containerID="08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123499    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="prometheus"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123504    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="prometheus"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123507    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27"} err="failed to get container status \"08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27\": rpc error: code = NotFound desc = could not find container \"08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27\": container with ID starting with 08ca5eecc56ddfe6afad9fda67561fe38d7297b546349f12f67fac84e8528a27 not found: ID does not exist"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123515    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-web"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123520    2565 scope.go:117] "RemoveContainer" containerID="6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123521    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-web"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123572    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-thanos"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123579    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-thanos"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123588    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" containerName="console"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123594    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" containerName="console"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123661    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="config-reloader"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123670    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="prometheus"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123676    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123682    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c55dc732-3e0c-497a-8dcc-4b0c7e21d9b3" containerName="console"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123687    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="thanos-sidecar"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123696    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-web"
Jul 16 04:56:22.123704 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123705    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" containerName="kube-rbac-proxy-thanos"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.123786    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f\": container with ID starting with 6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f not found: ID does not exist" containerID="6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123810    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f"} err="failed to get container status \"6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f\": rpc error: code = NotFound desc = could not find container \"6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f\": container with ID starting with 6cc64677afa2ff6303590a71c4387e00e6a2ce9cfe1e65507f1b287a5b58904f not found: ID does not exist"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.123844    2565 scope.go:117] "RemoveContainer" containerID="586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.124063    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c\": container with ID starting with 586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c not found: ID does not exist" containerID="586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.124080    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c"} err="failed to get container status \"586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c\": rpc error: code = NotFound desc = could not find container \"586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c\": container with ID starting with 586730c4dc69e26ae18922af522aee69baaefd376b902480b4420496d16e793c not found: ID does not exist"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.124093    2565 scope.go:117] "RemoveContainer" containerID="21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.124290    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea\": container with ID starting with 21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea not found: ID does not exist" containerID="21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.124324    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea"} err="failed to get container status \"21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea\": rpc error: code = NotFound desc = could not find container \"21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea\": container with ID starting with 21ed149674f1055913347e6bfcd772942f227d9e40736a2a0a962d1eb257d3ea not found: ID does not exist"
Jul 16 04:56:22.124388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.124337    2565 scope.go:117] "RemoveContainer" containerID="571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315"
Jul 16 04:56:22.124696 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.124576    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315\": container with ID starting with 571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315 not found: ID does not exist" containerID="571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315"
Jul 16 04:56:22.124696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.124593    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315"} err="failed to get container status \"571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315\": rpc error: code = NotFound desc = could not find container \"571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315\": container with ID starting with 571621147c560e2329dec3a2f89a23d8511ace46825c1ad2f29e2bb6dc101315 not found: ID does not exist"
Jul 16 04:56:22.124696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.124605    2565 scope.go:117] "RemoveContainer" containerID="cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4"
Jul 16 04:56:22.124994 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:56:22.124976    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4\": container with ID starting with cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4 not found: ID does not exist" containerID="cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4"
Jul 16 04:56:22.125055 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.125001    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4"} err="failed to get container status \"cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4\": rpc error: code = NotFound desc = could not find container \"cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4\": container with ID starting with cf32b4892b934ebd402f7ffbba472840d61e2f4d49b4a8d68cd3f0cb876301e4 not found: ID does not exist"
Jul 16 04:56:22.128507 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.128492    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.131172 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131155    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 04:56:22.131247 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131223    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 04:56:22.131290 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131252    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131519    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-l4h5v\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131547    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131556    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131564    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131583    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131553    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 04:56:22.131712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131520    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 04:56:22.132101 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131741    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-87dqm17ldf01k\""
Jul 16 04:56:22.132101 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.131742    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 04:56:22.132101 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.132016    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 04:56:22.132101 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.132025    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 04:56:22.132101 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.132046    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 04:56:22.134808 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.134784    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 04:56:22.137672 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.137651    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 04:56:22.142380 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.142361    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:56:22.284841 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.284799    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.284963 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.284847    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.284963 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.284876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/255c40e1-50f8-4583-ba71-edfd9b3393a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.284963 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.284912    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.284963 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.284929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.284963 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.284947    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-config\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285044    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285121 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285082    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285229 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285133    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285229 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285164    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285229 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285186    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285229 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285214    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285242    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285264    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzwpj\" (UniqueName: \"kubernetes.io/projected/255c40e1-50f8-4583-ba71-edfd9b3393a8-kube-api-access-zzwpj\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/255c40e1-50f8-4583-ba71-edfd9b3393a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285299    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.285474 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.285344    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386439 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386413    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386539 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386539 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386539 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386482    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386539 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386499    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386687 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386548    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386687 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386595    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.386687 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.386621    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387147 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387245 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzwpj\" (UniqueName: \"kubernetes.io/projected/255c40e1-50f8-4583-ba71-edfd9b3393a8-kube-api-access-zzwpj\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387245 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387210    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/255c40e1-50f8-4583-ba71-edfd9b3393a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387243    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387268    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387291    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387351 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387361    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/255c40e1-50f8-4583-ba71-edfd9b3393a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387414    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387463    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.387745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387487    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-config\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.388002 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.389572 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.389540    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.389665 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.389570    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.389665 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.387360    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.390211 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.389922    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.390211 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.390011    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.390211 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.390203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.390634 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.390609    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.390709 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.390612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-config\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.391417 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.391391    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.391632 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.391609    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.391737 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.391643    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/255c40e1-50f8-4583-ba71-edfd9b3393a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.391806 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.391776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.392388 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.392367    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/255c40e1-50f8-4583-ba71-edfd9b3393a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.392441 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.392376    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.392635 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.392618    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/255c40e1-50f8-4583-ba71-edfd9b3393a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.392995 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.392969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.393129 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.393111    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/255c40e1-50f8-4583-ba71-edfd9b3393a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.396894 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.396878    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzwpj\" (UniqueName: \"kubernetes.io/projected/255c40e1-50f8-4583-ba71-edfd9b3393a8-kube-api-access-zzwpj\") pod \"prometheus-k8s-0\" (UID: \"255c40e1-50f8-4583-ba71-edfd9b3393a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.444077 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.444055    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:22.577491 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:22.577385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 04:56:22.579901 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:56:22.579877    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255c40e1_50f8_4583_ba71_edfd9b3393a8.slice/crio-f80b0c98b1f32af828b704586845d32a2d0ee8be1b65581f02b9428ae2670817 WatchSource:0}: Error finding container f80b0c98b1f32af828b704586845d32a2d0ee8be1b65581f02b9428ae2670817: Status 404 returned error can't find the container with id f80b0c98b1f32af828b704586845d32a2d0ee8be1b65581f02b9428ae2670817
Jul 16 04:56:23.081053 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:23.081018    2565 generic.go:356] "Generic (PLEG): container finished" podID="255c40e1-50f8-4583-ba71-edfd9b3393a8" containerID="184be85720b41cc3e9ae7d139d03217d6be47613c303b13a5a9bb61868257339" exitCode=0
Jul 16 04:56:23.081194 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:23.081107    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerDied","Data":"184be85720b41cc3e9ae7d139d03217d6be47613c303b13a5a9bb61868257339"}
Jul 16 04:56:23.081194 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:23.081181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"f80b0c98b1f32af828b704586845d32a2d0ee8be1b65581f02b9428ae2670817"}
Jul 16 04:56:23.649341 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:23.649267    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9725736f-7fb3-4667-91e4-9bb30fce52fe" path="/var/lib/kubelet/pods/9725736f-7fb3-4667-91e4-9bb30fce52fe/volumes"
Jul 16 04:56:23.663253 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:23.663223    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:23.714706 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:23.714680    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:24.090156 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.090119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"ce9479291106ae25c49c8f99caad1ff473370e2a342860df7bef23db7b6d6ad5"}
Jul 16 04:56:24.090288 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.090204    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"b68435cd23312252e3bb6fd83ba8f3566b2b8d8865a26832152a5aa9b55dd052"}
Jul 16 04:56:24.090288 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.090228    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"2ff899de0d26de2bf12bc72cdf1283a1286aeac31649bc85edc337e66867dbd5"}
Jul 16 04:56:24.090288 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.090252    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"d8892b43ecac972f301d5cafd8056a9d372f1d406ea631de63f3f19b5af408eb"}
Jul 16 04:56:24.090288 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.090273    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"01f083879d06d6b5c6bc4dfda8bd0aade32a9e3f59b71227620904b2274f3d32"}
Jul 16 04:56:24.090493 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.090293    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"255c40e1-50f8-4583-ba71-edfd9b3393a8","Type":"ContainerStarted","Data":"dabf7e583decc127130fa8dd3e18044acadc370eb85ab1796353d85cce3b0639"}
Jul 16 04:56:24.118572 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:24.118533    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.09736116 podStartE2EDuration="2.118519869s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.158709ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:56:24.116485606 +0000 UTC m=+381.037068401" watchObservedRunningTime="2026-07-16 04:56:24.118519869 +0000 UTC m=+381.039102662"
Jul 16 04:56:25.449660 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:25.449630    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:25.549796 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:25.549770    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:26.789123 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:26.789094    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:26.843177 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:26.843149    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:26.900063 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:26.900033    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:27.444745 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:27.444717    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:56:31.846250 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:31.846212    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:31.949565 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:31.949540    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:56:31.996533 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:56:31.996504    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:07.365543 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:07.365507    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:13.890088 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:13.890052    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:22.444625 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:22.444591    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:57:22.459979 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:22.459957    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:57:22.881777 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:22.881747    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:22.932232 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:22.932206    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:23.324114 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:23.324090    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 04:57:31.564024 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:31.563944    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:39.724067 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:39.724025    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:57:39.823268 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:57:39.823235    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:01.261010 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:01.260982    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:02.076317 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:02.076283    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:04.870698 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:04.870665    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:04.970853 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:04.970800    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:09.663066 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:09.663033    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:09.710344 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:09.710316    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:11.617953 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:11.617918    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:11.715527 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:11.715495    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:12.138989 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:12.138956    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:12.241876 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:12.241821    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:13.615085 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:13.615056    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:14.007128 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:14.007046    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:22.421597 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:22.421565    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:23.050496 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:23.050458    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:23.098075 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:23.098055    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:24.562972 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:24.562938    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:24.662674 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:24.662648    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:27.968071 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:27.968037    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:28.021806 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:28.021780    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:29.965472 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:29.965444    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:30.063738 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:30.063714    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:32.172546 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:32.172501    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:48.106696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:48.106660    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:48.197920 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:48.197896    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:48.754793 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:48.754764    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:48.977772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:48.977742    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:49.081130 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.081107    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:49.128080 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.128054    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"]
Jul 16 04:58:49.131286 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.131271    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.133857 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.133841    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 04:58:49.133958 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.133870    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 04:58:49.133958 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.133940    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4rlvf\""
Jul 16 04:58:49.134066 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.133995    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 04:58:49.134250 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.134236    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 04:58:49.147805 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.147753    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4f7a532-4aef-4f87-8638-90a4a132d7ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.147930 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.147812    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4f7a532-4aef-4f87-8638-90a4a132d7ac-manifests\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.147930 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.147914    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtd9l\" (UniqueName: \"kubernetes.io/projected/d4f7a532-4aef-4f87-8638-90a4a132d7ac-kube-api-access-jtd9l\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.148033 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.147967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4f7a532-4aef-4f87-8638-90a4a132d7ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.148089 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.148053    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4f7a532-4aef-4f87-8638-90a4a132d7ac-tmp\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.148734 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.148717    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"]
Jul 16 04:58:49.249376 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.249351    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4f7a532-4aef-4f87-8638-90a4a132d7ac-tmp\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.249504 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.249382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4f7a532-4aef-4f87-8638-90a4a132d7ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.249504 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.249411    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4f7a532-4aef-4f87-8638-90a4a132d7ac-manifests\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.249504 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.249432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtd9l\" (UniqueName: \"kubernetes.io/projected/d4f7a532-4aef-4f87-8638-90a4a132d7ac-kube-api-access-jtd9l\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.249655 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.249544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4f7a532-4aef-4f87-8638-90a4a132d7ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.249867 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.249817    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d4f7a532-4aef-4f87-8638-90a4a132d7ac-manifests\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.251740 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.251719    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4f7a532-4aef-4f87-8638-90a4a132d7ac-tmp\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.251923 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.251817    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:49.252049 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.252023    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4f7a532-4aef-4f87-8638-90a4a132d7ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.252109 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.252052    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d4f7a532-4aef-4f87-8638-90a4a132d7ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.258752 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.258731    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtd9l\" (UniqueName: \"kubernetes.io/projected/d4f7a532-4aef-4f87-8638-90a4a132d7ac-kube-api-access-jtd9l\") pod \"opendatahub-operator-controller-manager-6dddb677c8-fl2lj\" (UID: \"d4f7a532-4aef-4f87-8638-90a4a132d7ac\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.351966 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.351910    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:49.441780 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.441750    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:49.451671 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.451646    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:49.552062 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.552030    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:49.578009 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.577985    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"]
Jul 16 04:58:49.579720 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:58:49.579697    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f7a532_4aef_4f87_8638_90a4a132d7ac.slice/crio-ddce732bb0f64128b971037e0d81c4854cf69ffa0e7c18b715f3fe33fc7980ab WatchSource:0}: Error finding container ddce732bb0f64128b971037e0d81c4854cf69ffa0e7c18b715f3fe33fc7980ab: Status 404 returned error can't find the container with id ddce732bb0f64128b971037e0d81c4854cf69ffa0e7c18b715f3fe33fc7980ab
Jul 16 04:58:49.609635 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.609580    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj" event={"ID":"d4f7a532-4aef-4f87-8638-90a4a132d7ac","Type":"ContainerStarted","Data":"ddce732bb0f64128b971037e0d81c4854cf69ffa0e7c18b715f3fe33fc7980ab"}
Jul 16 04:58:49.655870 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:49.655822    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:58:52.622799 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:52.622772    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj" event={"ID":"d4f7a532-4aef-4f87-8638-90a4a132d7ac","Type":"ContainerStarted","Data":"a667124535b8fa521938c0cbd068d4537561220e49ceea5804c66510334a66cd"}
Jul 16 04:58:53.628186 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:53.628155    2565 generic.go:356] "Generic (PLEG): container finished" podID="d4f7a532-4aef-4f87-8638-90a4a132d7ac" containerID="a667124535b8fa521938c0cbd068d4537561220e49ceea5804c66510334a66cd" exitCode=0
Jul 16 04:58:53.628584 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:53.628223    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj" event={"ID":"d4f7a532-4aef-4f87-8638-90a4a132d7ac","Type":"ContainerDied","Data":"a667124535b8fa521938c0cbd068d4537561220e49ceea5804c66510334a66cd"}
Jul 16 04:58:54.633772 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:54.633700    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj" event={"ID":"d4f7a532-4aef-4f87-8638-90a4a132d7ac","Type":"ContainerStarted","Data":"5ad04ea2735c6ed89ab06871bc397828055e89860645b6b7abd01e021bda1f6f"}
Jul 16 04:58:54.634132 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:54.633963    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:58:54.653206 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:58:54.653163    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj" podStartSLOduration=1.123300463 podStartE2EDuration="5.653150246s" totalImagesPullingTime="3.552737694s" totalInitContainerRuntime="977.112089ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:58:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:58:54.651462255 +0000 UTC m=+531.572045053" watchObservedRunningTime="2026-07-16 04:58:54.653150246 +0000 UTC m=+531.573733039"
Jul 16 04:59:00.948464 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:00.948429    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:01.042906 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:01.042880    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:04.515987 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:04.515954    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:04.614505 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:04.614477    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:05.127736 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:05.127705    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:05.214286 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:05.214256    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:05.640329 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:05.640302    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-fl2lj"
Jul 16 04:59:06.959193 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:06.959154    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:08.050210 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.050183    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:08.202877 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.202813    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:08.504384 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.504354    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:08.673698 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.673661    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:08.707394 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.707369    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"]
Jul 16 04:59:08.716012 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.715992    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.718276 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.718257    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 04:59:08.719087 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.719065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 04:59:08.719206 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.719177    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8nd5d\""
Jul 16 04:59:08.724215 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.724194    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"]
Jul 16 04:59:08.797027 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.796959    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-tmp\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.797027 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.797004    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-tls-certs\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.797186 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.797054    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7mk\" (UniqueName: \"kubernetes.io/projected/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-kube-api-access-6v7mk\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.887907 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.887883    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-996t6"]
Jul 16 04:59:08.890059 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.890046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:08.893161 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.893134    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lbcsk\""
Jul 16 04:59:08.893255 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.893184    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 16 04:59:08.897533 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.897514    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-tmp\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.897615 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.897545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-tls-certs\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.897615 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.897569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7mk\" (UniqueName: \"kubernetes.io/projected/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-kube-api-access-6v7mk\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.900004 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.899986    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-tmp\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.900076 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.900043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-tls-certs\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.902623 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.902605    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-996t6"]
Jul 16 04:59:08.909516 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.909489    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7mk\" (UniqueName: \"kubernetes.io/projected/ed30b757-2ec4-44e4-87ce-8f0c369b6b06-kube-api-access-6v7mk\") pod \"kube-auth-proxy-7c7fc4f54f-4vwml\" (UID: \"ed30b757-2ec4-44e4-87ce-8f0c369b6b06\") " pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:08.932536 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.932500    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:08.998719 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.998689    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxj6h\" (UniqueName: \"kubernetes.io/projected/cfa90ba6-1307-4694-8990-a7307b29936c-kube-api-access-kxj6h\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:08.998719 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:08.998721    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfa90ba6-1307-4694-8990-a7307b29936c-cert\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.026065 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.026029    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"
Jul 16 04:59:09.035125 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.035081    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:09.099594 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.099559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxj6h\" (UniqueName: \"kubernetes.io/projected/cfa90ba6-1307-4694-8990-a7307b29936c-kube-api-access-kxj6h\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.099955 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.099629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfa90ba6-1307-4694-8990-a7307b29936c-cert\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.099955 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:09.099756    2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 16 04:59:09.099955 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:09.099883    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfa90ba6-1307-4694-8990-a7307b29936c-cert podName:cfa90ba6-1307-4694-8990-a7307b29936c nodeName:}" failed. No retries permitted until 2026-07-16 04:59:09.599804589 +0000 UTC m=+546.520387361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cfa90ba6-1307-4694-8990-a7307b29936c-cert") pod "odh-model-controller-7998c67dc4-996t6" (UID: "cfa90ba6-1307-4694-8990-a7307b29936c") : secret "odh-model-controller-webhook-cert" not found
Jul 16 04:59:09.111959 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.111912    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxj6h\" (UniqueName: \"kubernetes.io/projected/cfa90ba6-1307-4694-8990-a7307b29936c-kube-api-access-kxj6h\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.136106 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.136059    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:09.156616 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.156595    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml"]
Jul 16 04:59:09.158267 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:59:09.158243    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded30b757_2ec4_44e4_87ce_8f0c369b6b06.slice/crio-c1ecd5b9815ceb02702d84badafcb3ff51ee9dfe73a65abb2a48a082788fac03 WatchSource:0}: Error finding container c1ecd5b9815ceb02702d84badafcb3ff51ee9dfe73a65abb2a48a082788fac03: Status 404 returned error can't find the container with id c1ecd5b9815ceb02702d84badafcb3ff51ee9dfe73a65abb2a48a082788fac03
Jul 16 04:59:09.534150 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.534086    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:09.605210 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.605182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfa90ba6-1307-4694-8990-a7307b29936c-cert\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.607576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.607559    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfa90ba6-1307-4694-8990-a7307b29936c-cert\") pod \"odh-model-controller-7998c67dc4-996t6\" (UID: \"cfa90ba6-1307-4694-8990-a7307b29936c\") " pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.684509 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.684484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml" event={"ID":"ed30b757-2ec4-44e4-87ce-8f0c369b6b06","Type":"ContainerStarted","Data":"c1ecd5b9815ceb02702d84badafcb3ff51ee9dfe73a65abb2a48a082788fac03"}
Jul 16 04:59:09.711033 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.711004    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:09.756281 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.756250    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gjj6v"]
Jul 16 04:59:09.760177 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.760155    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:09.762981 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.762960    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 16 04:59:09.763099 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.762968    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9fxxd\""
Jul 16 04:59:09.768576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.768550    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gjj6v"]
Jul 16 04:59:09.799168 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.798992    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:09.800390 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.800364    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"]
Jul 16 04:59:09.803095 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.803075    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.804374 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.804351    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:09.805617 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.805535    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 04:59:09.805617 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.805559    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 04:59:09.805617 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.805598    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 04:59:09.805818 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.805559    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 04:59:09.805918 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.805902    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wd4vg\""
Jul 16 04:59:09.806428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.806346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-tls-certs\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:09.806428 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.806398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgx2g\" (UniqueName: \"kubernetes.io/projected/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-kube-api-access-zgx2g\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:09.816976 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.816931    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"]
Jul 16 04:59:09.907412 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907380    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-istio-token\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.907551 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.907551 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-tls-certs\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:09.907551 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907506    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d661014-452a-4733-97c4-b7a53a004316-local-certs\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.907712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907548    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.907712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907578    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p77hf\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-kube-api-access-p77hf\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.907712 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:09.907606    2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 16 04:59:09.907712 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgx2g\" (UniqueName: \"kubernetes.io/projected/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-kube-api-access-zgx2g\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:09.907712 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:09.907665    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-tls-certs podName:2fdf5f77-c3d2-46e4-bc31-42b39a128e30 nodeName:}" failed. No retries permitted until 2026-07-16 04:59:10.407644051 +0000 UTC m=+547.328226825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-tls-certs") pod "model-serving-api-54d89bd5fc-gjj6v" (UID: "2fdf5f77-c3d2-46e4-bc31-42b39a128e30") : secret "model-serving-api-tls" not found
Jul 16 04:59:09.907994 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907912    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.907994 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.907986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-cacerts\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:09.918467 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.918401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgx2g\" (UniqueName: \"kubernetes.io/projected/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-kube-api-access-zgx2g\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:09.924997 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.924972    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:09.952275 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:09.952246    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-996t6"]
Jul 16 04:59:09.956333 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:59:09.956305    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa90ba6_1307_4694_8990_a7307b29936c.slice/crio-73bb45676030dd1ca6a693a97242b1843d0dc23c30092b7626efaaabdbe78c9a WatchSource:0}: Error finding container 73bb45676030dd1ca6a693a97242b1843d0dc23c30092b7626efaaabdbe78c9a: Status 404 returned error can't find the container with id 73bb45676030dd1ca6a693a97242b1843d0dc23c30092b7626efaaabdbe78c9a
Jul 16 04:59:10.009440 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d661014-452a-4733-97c4-b7a53a004316-local-certs\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.009600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009449    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.009600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p77hf\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-kube-api-access-p77hf\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.009600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.009600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-cacerts\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.009600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009595    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-istio-token\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.009950 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.009619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.010337 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.010214    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.012453 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.012389    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.012453 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.012395    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-cacerts\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.012618 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.012595    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d661014-452a-4733-97c4-b7a53a004316-local-certs\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.012664 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.012650    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.018608 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.018577    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p77hf\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-kube-api-access-p77hf\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.018718 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.018606    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-istio-token\") pod \"istiod-openshift-gateway-7574459656-7jnn4\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.115810 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.115785    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:10.202501 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.202475    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:10.261875 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.261811    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"]
Jul 16 04:59:10.264016 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:59:10.263988    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d661014_452a_4733_97c4_b7a53a004316.slice/crio-79ccefe64df6cff682e9eb7db2fa336336b801c5e0cafec9bb926849ee32564f WatchSource:0}: Error finding container 79ccefe64df6cff682e9eb7db2fa336336b801c5e0cafec9bb926849ee32564f: Status 404 returned error can't find the container with id 79ccefe64df6cff682e9eb7db2fa336336b801c5e0cafec9bb926849ee32564f
Jul 16 04:59:10.413951 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.413869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-tls-certs\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:10.416613 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.416581    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fdf5f77-c3d2-46e4-bc31-42b39a128e30-tls-certs\") pod \"model-serving-api-54d89bd5fc-gjj6v\" (UID: \"2fdf5f77-c3d2-46e4-bc31-42b39a128e30\") " pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:10.675877 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.675776    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:10.690651 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.690616    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-996t6" event={"ID":"cfa90ba6-1307-4694-8990-a7307b29936c","Type":"ContainerStarted","Data":"73bb45676030dd1ca6a693a97242b1843d0dc23c30092b7626efaaabdbe78c9a"}
Jul 16 04:59:10.693033 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.693002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4" event={"ID":"4d661014-452a-4733-97c4-b7a53a004316","Type":"ContainerStarted","Data":"79ccefe64df6cff682e9eb7db2fa336336b801c5e0cafec9bb926849ee32564f"}
Jul 16 04:59:10.776129 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.776061    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:10.827616 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:10.827581    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gjj6v"]
Jul 16 04:59:10.831817 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:59:10.831769    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdf5f77_c3d2_46e4_bc31_42b39a128e30.slice/crio-9930aeedf5bc3a8b2a70885db53816e336bbff0cf638426bebb0e3d72daba976 WatchSource:0}: Error finding container 9930aeedf5bc3a8b2a70885db53816e336bbff0cf638426bebb0e3d72daba976: Status 404 returned error can't find the container with id 9930aeedf5bc3a8b2a70885db53816e336bbff0cf638426bebb0e3d72daba976
Jul 16 04:59:11.701339 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:11.701298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v" event={"ID":"2fdf5f77-c3d2-46e4-bc31-42b39a128e30","Type":"ContainerStarted","Data":"9930aeedf5bc3a8b2a70885db53816e336bbff0cf638426bebb0e3d72daba976"}
Jul 16 04:59:12.752214 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:12.752176    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:12.800081 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:12.800035    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:14.587907 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.587875    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 16 04:59:14.588192 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.587942    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 16 04:59:14.715222 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.715092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml" event={"ID":"ed30b757-2ec4-44e4-87ce-8f0c369b6b06","Type":"ContainerStarted","Data":"9d0dcbc490c5aaf56201f772b059a10ec43aa48a701f9e92e95affa1c8bb201d"}
Jul 16 04:59:14.717197 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.717154    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v" event={"ID":"2fdf5f77-c3d2-46e4-bc31-42b39a128e30","Type":"ContainerStarted","Data":"a80efbf5cbdaca73203bf93baad8070f84548928eb35f5ee7eee1fddeecfc5fb"}
Jul 16 04:59:14.717888 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.717622    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:14.719404 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.719378    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4" event={"ID":"4d661014-452a-4733-97c4-b7a53a004316","Type":"ContainerStarted","Data":"cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279"}
Jul 16 04:59:14.719513 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.719429    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:14.721131 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.721111    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-996t6" event={"ID":"cfa90ba6-1307-4694-8990-a7307b29936c","Type":"ContainerStarted","Data":"645638364fb1633deeae6e5f7e373faaf43a9bc93e99aad2f51eb20c978dead8"}
Jul 16 04:59:14.721238 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.721227    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:14.748004 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.747948    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c7fc4f54f-4vwml" podStartSLOduration=1.32955572 podStartE2EDuration="6.747934476s" totalImagesPullingTime="5.418378756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:14.745538748 +0000 UTC m=+551.666121565" watchObservedRunningTime="2026-07-16 04:59:14.747934476 +0000 UTC m=+551.668517269"
Jul 16 04:59:14.772700 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.772660    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-996t6" podStartSLOduration=2.145623951 podStartE2EDuration="6.772647083s" totalImagesPullingTime="4.627023132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:14.77147237 +0000 UTC m=+551.692055164" watchObservedRunningTime="2026-07-16 04:59:14.772647083 +0000 UTC m=+551.693229876"
Jul 16 04:59:14.798568 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.798517    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v" podStartSLOduration=2.043254938 podStartE2EDuration="5.798503801s" totalImagesPullingTime="3.755248863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:14.797815596 +0000 UTC m=+551.718398389" watchObservedRunningTime="2026-07-16 04:59:14.798503801 +0000 UTC m=+551.719086595"
Jul 16 04:59:14.818730 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:14.818692    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4" podStartSLOduration=1.497383695 podStartE2EDuration="5.818675435s" totalImagesPullingTime="4.32129174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:14.817968695 +0000 UTC m=+551.738551488" watchObservedRunningTime="2026-07-16 04:59:14.818675435 +0000 UTC m=+551.739258228"
Jul 16 04:59:15.208560 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:15.208460    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:15.343110 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:15.343073    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:15.478214 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:15.478121    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:15.650521 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:15.650493    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:15.727855 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:15.727807    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:15.933457 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:15.933431    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:18.394225 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.394189    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:18.438950 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.438921    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"]
Jul 16 04:59:18.442600 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.442579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:18.445085 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.445065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 16 04:59:18.445218 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.445077    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vp524\""
Jul 16 04:59:18.453654 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.453631    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"]
Jul 16 04:59:18.454879 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.454850    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"]
Jul 16 04:59:18.458054 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.458034    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.460349 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.460330    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 16 04:59:18.460454 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.460391    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6pw7v\""
Jul 16 04:59:18.469477 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.469452    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"]
Jul 16 04:59:18.475038 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.475014    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:18.530507 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.530480    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:18.590655 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.590628    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwtbc\" (UniqueName: \"kubernetes.io/projected/4e348d8d-fc1c-40e5-96d4-9cdf11326572-kube-api-access-vwtbc\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:18.590743 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.590664    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821c370a-1f47-4acb-a1ab-64e407151fae-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5lkgb\" (UID: \"821c370a-1f47-4acb-a1ab-64e407151fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.590743 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.590710    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e348d8d-fc1c-40e5-96d4-9cdf11326572-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:18.590854 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.590768    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqs8\" (UniqueName: \"kubernetes.io/projected/821c370a-1f47-4acb-a1ab-64e407151fae-kube-api-access-8lqs8\") pod \"llmisvc-controller-manager-777c6bd7d5-5lkgb\" (UID: \"821c370a-1f47-4acb-a1ab-64e407151fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.691564 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.691500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e348d8d-fc1c-40e5-96d4-9cdf11326572-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:18.691564 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.691558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqs8\" (UniqueName: \"kubernetes.io/projected/821c370a-1f47-4acb-a1ab-64e407151fae-kube-api-access-8lqs8\") pod \"llmisvc-controller-manager-777c6bd7d5-5lkgb\" (UID: \"821c370a-1f47-4acb-a1ab-64e407151fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.691696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.691598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwtbc\" (UniqueName: \"kubernetes.io/projected/4e348d8d-fc1c-40e5-96d4-9cdf11326572-kube-api-access-vwtbc\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:18.691696 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.691634    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821c370a-1f47-4acb-a1ab-64e407151fae-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5lkgb\" (UID: \"821c370a-1f47-4acb-a1ab-64e407151fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.691696 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:18.691656    2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 16 04:59:18.691859 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:18.691716    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e348d8d-fc1c-40e5-96d4-9cdf11326572-cert podName:4e348d8d-fc1c-40e5-96d4-9cdf11326572 nodeName:}" failed. No retries permitted until 2026-07-16 04:59:19.191697558 +0000 UTC m=+556.112280335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4e348d8d-fc1c-40e5-96d4-9cdf11326572-cert") pod "kserve-controller-manager-d6f4b65b8-rh6st" (UID: "4e348d8d-fc1c-40e5-96d4-9cdf11326572") : secret "kserve-webhook-server-cert" not found
Jul 16 04:59:18.694188 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.694168    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821c370a-1f47-4acb-a1ab-64e407151fae-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5lkgb\" (UID: \"821c370a-1f47-4acb-a1ab-64e407151fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.707080 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.707057    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqs8\" (UniqueName: \"kubernetes.io/projected/821c370a-1f47-4acb-a1ab-64e407151fae-kube-api-access-8lqs8\") pod \"llmisvc-controller-manager-777c6bd7d5-5lkgb\" (UID: \"821c370a-1f47-4acb-a1ab-64e407151fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.712982 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.712960    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwtbc\" (UniqueName: \"kubernetes.io/projected/4e348d8d-fc1c-40e5-96d4-9cdf11326572-kube-api-access-vwtbc\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:18.767544 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.767518    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:18.877948 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.877910    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:18.899930 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:18.899910    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"]
Jul 16 04:59:18.902106 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:59:18.902076    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod821c370a_1f47_4acb_a1ab_64e407151fae.slice/crio-e23b945b4ea0e1051411778d79af308614c0cb4e3dc66e27e7ee45c524e16109 WatchSource:0}: Error finding container e23b945b4ea0e1051411778d79af308614c0cb4e3dc66e27e7ee45c524e16109: Status 404 returned error can't find the container with id e23b945b4ea0e1051411778d79af308614c0cb4e3dc66e27e7ee45c524e16109
Jul 16 04:59:19.197233 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.197195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e348d8d-fc1c-40e5-96d4-9cdf11326572-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:19.199576 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.199552    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e348d8d-fc1c-40e5-96d4-9cdf11326572-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh6st\" (UID: \"4e348d8d-fc1c-40e5-96d4-9cdf11326572\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:19.352469 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.352443    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:19.481507 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.481437    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"]
Jul 16 04:59:19.484116 ip-10-0-138-249 kubenswrapper[2565]: W0716 04:59:19.484090    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e348d8d_fc1c_40e5_96d4_9cdf11326572.slice/crio-fe9c0096d0eec6a8a894836239a4ea2ace4491f805fdafb0f0cd27dac1c7febd WatchSource:0}: Error finding container fe9c0096d0eec6a8a894836239a4ea2ace4491f805fdafb0f0cd27dac1c7febd: Status 404 returned error can't find the container with id fe9c0096d0eec6a8a894836239a4ea2ace4491f805fdafb0f0cd27dac1c7febd
Jul 16 04:59:19.673907 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.673882    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:19.744496 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.744427    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb" event={"ID":"821c370a-1f47-4acb-a1ab-64e407151fae","Type":"ContainerStarted","Data":"e23b945b4ea0e1051411778d79af308614c0cb4e3dc66e27e7ee45c524e16109"}
Jul 16 04:59:19.745748 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.745723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st" event={"ID":"4e348d8d-fc1c-40e5-96d4-9cdf11326572","Type":"ContainerStarted","Data":"fe9c0096d0eec6a8a894836239a4ea2ace4491f805fdafb0f0cd27dac1c7febd"}
Jul 16 04:59:19.973380 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:19.973347    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:20.030334 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.030267    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:20.076272 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.076243    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"]
Jul 16 04:59:20.076469 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.076447    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4" podUID="4d661014-452a-4733-97c4-b7a53a004316" containerName="discovery" containerID="cri-o://cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279" gracePeriod=30
Jul 16 04:59:20.505972 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.505942    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:20.609028 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.608998    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-istio-token\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609040    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d661014-452a-4733-97c4-b7a53a004316-local-certs\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609120    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p77hf\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-kube-api-access-p77hf\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609151    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-dns-cert\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609216 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609208    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-ca-configmap\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609413 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609231    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-cacerts\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609413 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609292    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-kubeconfig\") pod \"4d661014-452a-4733-97c4-b7a53a004316\" (UID: \"4d661014-452a-4733-97c4-b7a53a004316\") "
Jul 16 04:59:20.609701 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.609669    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 04:59:20.611854 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.611751    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:59:20.612147 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.612117    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d661014-452a-4733-97c4-b7a53a004316-local-certs" (OuterVolumeSpecName: "local-certs") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 04:59:20.612242 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.612161    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-cacerts" (OuterVolumeSpecName: "cacerts") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:59:20.612309 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.612289    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:59:20.612506 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.612487    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 04:59:20.612506 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.612492    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-kube-api-access-p77hf" (OuterVolumeSpecName: "kube-api-access-p77hf") pod "4d661014-452a-4733-97c4-b7a53a004316" (UID: "4d661014-452a-4733-97c4-b7a53a004316"). InnerVolumeSpecName "kube-api-access-p77hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 04:59:20.710020 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.709968    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p77hf\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-kube-api-access-p77hf\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.710020 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.710018    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-dns-cert\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.710202 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.710036    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d661014-452a-4733-97c4-b7a53a004316-istio-csr-ca-configmap\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.710202 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.710051    2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-cacerts\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.710202 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.710064    2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d661014-452a-4733-97c4-b7a53a004316-istio-kubeconfig\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.710202 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.710077    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d661014-452a-4733-97c4-b7a53a004316-istio-token\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.710202 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.710091    2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d661014-452a-4733-97c4-b7a53a004316-local-certs\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 04:59:20.753607 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.753533    2565 generic.go:356] "Generic (PLEG): container finished" podID="4d661014-452a-4733-97c4-b7a53a004316" containerID="cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279" exitCode=0
Jul 16 04:59:20.753607 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.753576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4" event={"ID":"4d661014-452a-4733-97c4-b7a53a004316","Type":"ContainerDied","Data":"cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279"}
Jul 16 04:59:20.753607 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.753582    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"
Jul 16 04:59:20.753809 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.753625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4" event={"ID":"4d661014-452a-4733-97c4-b7a53a004316","Type":"ContainerDied","Data":"79ccefe64df6cff682e9eb7db2fa336336b801c5e0cafec9bb926849ee32564f"}
Jul 16 04:59:20.753809 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.753652    2565 scope.go:117] "RemoveContainer" containerID="cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279"
Jul 16 04:59:20.764595 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.764505    2565 scope.go:117] "RemoveContainer" containerID="cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279"
Jul 16 04:59:20.764952 ip-10-0-138-249 kubenswrapper[2565]: E0716 04:59:20.764886    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279\": container with ID starting with cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279 not found: ID does not exist" containerID="cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279"
Jul 16 04:59:20.764952 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.764921    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279"} err="failed to get container status \"cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279\": rpc error: code = NotFound desc = could not find container \"cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279\": container with ID starting with cde3b03ce8307e79e44c7ac9fa5a05d870d6867ce6e899bcaeb92a3e453d6279 not found: ID does not exist"
Jul 16 04:59:20.777267 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.777247    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"]
Jul 16 04:59:20.785087 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:20.785038    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-7jnn4"]
Jul 16 04:59:21.652252 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:21.652205    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d661014-452a-4733-97c4-b7a53a004316" path="/var/lib/kubelet/pods/4d661014-452a-4733-97c4-b7a53a004316/volumes"
Jul 16 04:59:22.764419 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:22.764302    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb" event={"ID":"821c370a-1f47-4acb-a1ab-64e407151fae","Type":"ContainerStarted","Data":"6d3ea888b771d6b71756085230f70f469f6b6395e22fbcb4ad2bd76f07903168"}
Jul 16 04:59:22.764419 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:22.764387    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:22.765778 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:22.765751    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st" event={"ID":"4e348d8d-fc1c-40e5-96d4-9cdf11326572","Type":"ContainerStarted","Data":"538d94c66487e0e97cd363e204435d3266d1a35b52f7ba5b3a74847897bf731d"}
Jul 16 04:59:22.765910 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:22.765881    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 04:59:22.781357 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:22.781313    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb" podStartSLOduration=1.225798468 podStartE2EDuration="4.781299768s" totalImagesPullingTime="3.5555013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:22.780019002 +0000 UTC m=+559.700601797" watchObservedRunningTime="2026-07-16 04:59:22.781299768 +0000 UTC m=+559.701882561"
Jul 16 04:59:22.798015 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:22.797964    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st" podStartSLOduration=1.828535649 podStartE2EDuration="4.797946764s" totalImagesPullingTime="2.969411115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:22.796136802 +0000 UTC m=+559.716719597" watchObservedRunningTime="2026-07-16 04:59:22.797946764 +0000 UTC m=+559.718529559"
Jul 16 04:59:23.870343 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:23.870312    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:23.971790 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:23.971750    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 04:59:25.728676 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:25.728647    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-996t6"
Jul 16 04:59:25.734969 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:25.734942    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gjj6v"
Jul 16 04:59:53.771540 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:53.771506    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5lkgb"
Jul 16 04:59:53.774362 ip-10-0-138-249 kubenswrapper[2565]: I0716 04:59:53.774340    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh6st"
Jul 16 05:00:04.197622 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:04.197513    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:00:04.202661 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:04.198255    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:00:09.814385 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.814355    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:09.818130 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.818107    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh9jl"]
Jul 16 05:00:09.818418 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.818408    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d661014-452a-4733-97c4-b7a53a004316" containerName="discovery"
Jul 16 05:00:09.818473 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.818420    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d661014-452a-4733-97c4-b7a53a004316" containerName="discovery"
Jul 16 05:00:09.818473 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.818468    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d661014-452a-4733-97c4-b7a53a004316" containerName="discovery"
Jul 16 05:00:09.821312 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.821295    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:09.823722 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.823701    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 05:00:09.823722 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.823714    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qj4pm\""
Jul 16 05:00:09.824369 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.824356    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 05:00:09.831849 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.831810    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh9jl"]
Jul 16 05:00:09.873226 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.873203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9brdz\" (UniqueName: \"kubernetes.io/projected/fe81a5f9-73be-48e6-b355-c989c750153c-kube-api-access-9brdz\") pod \"kuadrant-operator-catalog-gh9jl\" (UID: \"fe81a5f9-73be-48e6-b355-c989c750153c\") " pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:09.973627 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.973607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9brdz\" (UniqueName: \"kubernetes.io/projected/fe81a5f9-73be-48e6-b355-c989c750153c-kube-api-access-9brdz\") pod \"kuadrant-operator-catalog-gh9jl\" (UID: \"fe81a5f9-73be-48e6-b355-c989c750153c\") " pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:09.981321 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:09.981294    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9brdz\" (UniqueName: \"kubernetes.io/projected/fe81a5f9-73be-48e6-b355-c989c750153c-kube-api-access-9brdz\") pod \"kuadrant-operator-catalog-gh9jl\" (UID: \"fe81a5f9-73be-48e6-b355-c989c750153c\") " pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:10.130741 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.130715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:10.255113 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.255088    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh9jl"]
Jul 16 05:00:10.257231 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:10.257198    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe81a5f9_73be_48e6_b355_c989c750153c.slice/crio-664b552b749c845a809be4f26aed0564e11027e7452c193600d3e00deb865364 WatchSource:0}: Error finding container 664b552b749c845a809be4f26aed0564e11027e7452c193600d3e00deb865364: Status 404 returned error can't find the container with id 664b552b749c845a809be4f26aed0564e11027e7452c193600d3e00deb865364
Jul 16 05:00:10.258535 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.258515    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 05:00:10.435279 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.435208    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh9jl"]
Jul 16 05:00:10.450286 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.450261    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wx7rh"]
Jul 16 05:00:10.454457 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.454441    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:10.460115 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.460095    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wx7rh"]
Jul 16 05:00:10.478012 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.477984    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xpl8\" (UniqueName: \"kubernetes.io/projected/3c2e9023-2c5c-4152-92c2-3020c6073414-kube-api-access-7xpl8\") pod \"kuadrant-operator-catalog-wx7rh\" (UID: \"3c2e9023-2c5c-4152-92c2-3020c6073414\") " pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:10.579365 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.579342    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xpl8\" (UniqueName: \"kubernetes.io/projected/3c2e9023-2c5c-4152-92c2-3020c6073414-kube-api-access-7xpl8\") pod \"kuadrant-operator-catalog-wx7rh\" (UID: \"3c2e9023-2c5c-4152-92c2-3020c6073414\") " pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:10.588505 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.588481    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xpl8\" (UniqueName: \"kubernetes.io/projected/3c2e9023-2c5c-4152-92c2-3020c6073414-kube-api-access-7xpl8\") pod \"kuadrant-operator-catalog-wx7rh\" (UID: \"3c2e9023-2c5c-4152-92c2-3020c6073414\") " pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:10.764701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.764628    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:10.902561 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.902536    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wx7rh"]
Jul 16 05:00:10.904071 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:10.904044    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2e9023_2c5c_4152_92c2_3020c6073414.slice/crio-fb8968935d99d6a2dcc3e85becdf8f88d208e8a6fdf43a1ed24918801dbee0d0 WatchSource:0}: Error finding container fb8968935d99d6a2dcc3e85becdf8f88d208e8a6fdf43a1ed24918801dbee0d0: Status 404 returned error can't find the container with id fb8968935d99d6a2dcc3e85becdf8f88d208e8a6fdf43a1ed24918801dbee0d0
Jul 16 05:00:10.951555 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.951522    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh" event={"ID":"3c2e9023-2c5c-4152-92c2-3020c6073414","Type":"ContainerStarted","Data":"fb8968935d99d6a2dcc3e85becdf8f88d208e8a6fdf43a1ed24918801dbee0d0"}
Jul 16 05:00:10.952592 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:10.952571    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl" event={"ID":"fe81a5f9-73be-48e6-b355-c989c750153c","Type":"ContainerStarted","Data":"664b552b749c845a809be4f26aed0564e11027e7452c193600d3e00deb865364"}
Jul 16 05:00:12.056324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:12.056294    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:12.962798 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:12.962762    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh" event={"ID":"3c2e9023-2c5c-4152-92c2-3020c6073414","Type":"ContainerStarted","Data":"e3667cd76e7ec11ebdfc999a8911bbdcf5540c6dfe977c6924eb0035aed8e053"}
Jul 16 05:00:12.964258 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:12.964234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl" event={"ID":"fe81a5f9-73be-48e6-b355-c989c750153c","Type":"ContainerStarted","Data":"22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd"}
Jul 16 05:00:12.964381 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:12.964275    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl" podUID="fe81a5f9-73be-48e6-b355-c989c750153c" containerName="registry-server" containerID="cri-o://22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd" gracePeriod=2
Jul 16 05:00:12.977395 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:12.977350    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh" podStartSLOduration=1.402596585 podStartE2EDuration="2.977334214s" totalImagesPullingTime="1.574737629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:00:12.976316188 +0000 UTC m=+609.896898982" watchObservedRunningTime="2026-07-16 05:00:12.977334214 +0000 UTC m=+609.897917009"
Jul 16 05:00:12.990392 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:12.990354    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl" podStartSLOduration=1.769354952 podStartE2EDuration="3.990343038s" totalImagesPullingTime="2.220988086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:00:12.989266335 +0000 UTC m=+609.909849132" watchObservedRunningTime="2026-07-16 05:00:12.990343038 +0000 UTC m=+609.910925857"
Jul 16 05:00:13.198944 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.198921    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:13.305647 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.305586    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9brdz\" (UniqueName: \"kubernetes.io/projected/fe81a5f9-73be-48e6-b355-c989c750153c-kube-api-access-9brdz\") pod \"fe81a5f9-73be-48e6-b355-c989c750153c\" (UID: \"fe81a5f9-73be-48e6-b355-c989c750153c\") "
Jul 16 05:00:13.307702 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.307677    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe81a5f9-73be-48e6-b355-c989c750153c-kube-api-access-9brdz" (OuterVolumeSpecName: "kube-api-access-9brdz") pod "fe81a5f9-73be-48e6-b355-c989c750153c" (UID: "fe81a5f9-73be-48e6-b355-c989c750153c"). InnerVolumeSpecName "kube-api-access-9brdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:00:13.406721 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.406700    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9brdz\" (UniqueName: \"kubernetes.io/projected/fe81a5f9-73be-48e6-b355-c989c750153c-kube-api-access-9brdz\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:00:13.970014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.969945    2565 generic.go:356] "Generic (PLEG): container finished" podID="fe81a5f9-73be-48e6-b355-c989c750153c" containerID="22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd" exitCode=0
Jul 16 05:00:13.970014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.969992    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl"
Jul 16 05:00:13.970165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.970034    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl" event={"ID":"fe81a5f9-73be-48e6-b355-c989c750153c","Type":"ContainerDied","Data":"22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd"}
Jul 16 05:00:13.970165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.970085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gh9jl" event={"ID":"fe81a5f9-73be-48e6-b355-c989c750153c","Type":"ContainerDied","Data":"664b552b749c845a809be4f26aed0564e11027e7452c193600d3e00deb865364"}
Jul 16 05:00:13.970165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.970109    2565 scope.go:117] "RemoveContainer" containerID="22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd"
Jul 16 05:00:13.978284 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.978268    2565 scope.go:117] "RemoveContainer" containerID="22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd"
Jul 16 05:00:13.978557 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:00:13.978539    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd\": container with ID starting with 22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd not found: ID does not exist" containerID="22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd"
Jul 16 05:00:13.978611 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.978566    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd"} err="failed to get container status \"22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd\": rpc error: code = NotFound desc = could not find container \"22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd\": container with ID starting with 22f41378d07bc3bb066b63955c6586b2a4d8d2f3ab0cf8e57f9fb7014cf530cd not found: ID does not exist"
Jul 16 05:00:13.984425 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.984404    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh9jl"]
Jul 16 05:00:13.988287 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:13.988270    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh9jl"]
Jul 16 05:00:15.649902 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:15.649874    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe81a5f9-73be-48e6-b355-c989c750153c" path="/var/lib/kubelet/pods/fe81a5f9-73be-48e6-b355-c989c750153c/volumes"
Jul 16 05:00:20.765340 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:20.765314    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:20.765908 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:20.765353    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:20.787570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:20.787548    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:21.018604 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:21.018533    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-wx7rh"
Jul 16 05:00:36.937737 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.937705    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-778d57fdbc-g2hxr"]
Jul 16 05:00:36.938106 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.938021    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe81a5f9-73be-48e6-b355-c989c750153c" containerName="registry-server"
Jul 16 05:00:36.938106 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.938032    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe81a5f9-73be-48e6-b355-c989c750153c" containerName="registry-server"
Jul 16 05:00:36.938106 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.938088    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe81a5f9-73be-48e6-b355-c989c750153c" containerName="registry-server"
Jul 16 05:00:36.940911 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.940894    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:36.943674 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.943657    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 05:00:36.944476 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944453    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 05:00:36.944476 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944468    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 05:00:36.944664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944497    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 05:00:36.944664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944461    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4bwwt\""
Jul 16 05:00:36.944664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944462    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 05:00:36.944664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944463    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 05:00:36.945016 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.944992    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 05:00:36.948561 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.948543    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 05:00:36.953092 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:36.953074    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-778d57fdbc-g2hxr"]
Jul 16 05:00:37.003604 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.003579    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:37.077799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.077773    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-serving-cert\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.077799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.077802    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-oauth-serving-cert\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.078027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.077938    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-trusted-ca-bundle\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.078027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.077970    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-oauth-config\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.078129 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.078032    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-service-ca\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.078129 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.078060    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-config\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.078129 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.078091    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfmzt\" (UniqueName: \"kubernetes.io/projected/f6cd5411-d87a-45fe-976a-9404a1db72e5-kube-api-access-nfmzt\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.105762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.105734    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:37.178525 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-trusted-ca-bundle\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.178634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-oauth-config\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.178634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-service-ca\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.178634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178591    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-config\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.178634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfmzt\" (UniqueName: \"kubernetes.io/projected/f6cd5411-d87a-45fe-976a-9404a1db72e5-kube-api-access-nfmzt\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.178865 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-serving-cert\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.178865 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.178690    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-oauth-serving-cert\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.179349 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.179327    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-config\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.179433 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.179375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-service-ca\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.179433 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.179408    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-trusted-ca-bundle\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.179661 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.179640    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6cd5411-d87a-45fe-976a-9404a1db72e5-oauth-serving-cert\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.181009 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.180990    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-oauth-config\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.181205 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.181179    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6cd5411-d87a-45fe-976a-9404a1db72e5-console-serving-cert\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.187254 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.187235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfmzt\" (UniqueName: \"kubernetes.io/projected/f6cd5411-d87a-45fe-976a-9404a1db72e5-kube-api-access-nfmzt\") pod \"console-778d57fdbc-g2hxr\" (UID: \"f6cd5411-d87a-45fe-976a-9404a1db72e5\") " pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.208713 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.208657    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:37.250374 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.250344    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:37.377890 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.377853    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-778d57fdbc-g2hxr"]
Jul 16 05:00:37.380226 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:37.380193    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6cd5411_d87a_45fe_976a_9404a1db72e5.slice/crio-1ed78c8d19c06315522872ab0f067b56d2d1f63b4e9187091f88e79ca647e7e6 WatchSource:0}: Error finding container 1ed78c8d19c06315522872ab0f067b56d2d1f63b4e9187091f88e79ca647e7e6: Status 404 returned error can't find the container with id 1ed78c8d19c06315522872ab0f067b56d2d1f63b4e9187091f88e79ca647e7e6
Jul 16 05:00:37.768695 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.768661    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:37.875735 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:37.875709    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:38.068233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:38.068205    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778d57fdbc-g2hxr" event={"ID":"f6cd5411-d87a-45fe-976a-9404a1db72e5","Type":"ContainerStarted","Data":"5ed4346909e54dedda16b948dcf5e89d76ed160d5526ee32a11c749554bbce73"}
Jul 16 05:00:38.068645 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:38.068247    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-778d57fdbc-g2hxr" event={"ID":"f6cd5411-d87a-45fe-976a-9404a1db72e5","Type":"ContainerStarted","Data":"1ed78c8d19c06315522872ab0f067b56d2d1f63b4e9187091f88e79ca647e7e6"}
Jul 16 05:00:38.085849 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:38.085786    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-778d57fdbc-g2hxr" podStartSLOduration=2.085771167 podStartE2EDuration="2.085771167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:00:38.084398608 +0000 UTC m=+635.004981402" watchObservedRunningTime="2026-07-16 05:00:38.085771167 +0000 UTC m=+635.006353959"
Jul 16 05:00:43.057670 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:43.057640    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:46.078641 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.078610    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"]
Jul 16 05:00:46.086792 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.086774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:00:46.089179 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.089155    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 16 05:00:46.089267 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.089201    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qmdv7\""
Jul 16 05:00:46.092712 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.092695    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"]
Jul 16 05:00:46.148845 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.148807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flfgx\" (UniqueName: \"kubernetes.io/projected/f982aded-5030-4aad-b09e-9ecceba4450c-kube-api-access-flfgx\") pod \"dns-operator-controller-manager-864dd78c4c-jzjzd\" (UID: \"f982aded-5030-4aad-b09e-9ecceba4450c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:00:46.250012 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.249989    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flfgx\" (UniqueName: \"kubernetes.io/projected/f982aded-5030-4aad-b09e-9ecceba4450c-kube-api-access-flfgx\") pod \"dns-operator-controller-manager-864dd78c4c-jzjzd\" (UID: \"f982aded-5030-4aad-b09e-9ecceba4450c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:00:46.264509 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.264485    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flfgx\" (UniqueName: \"kubernetes.io/projected/f982aded-5030-4aad-b09e-9ecceba4450c-kube-api-access-flfgx\") pod \"dns-operator-controller-manager-864dd78c4c-jzjzd\" (UID: \"f982aded-5030-4aad-b09e-9ecceba4450c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:00:46.398842 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.398755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:00:46.529233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:46.529203    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"]
Jul 16 05:00:46.531809 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:46.531777    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf982aded_5030_4aad_b09e_9ecceba4450c.slice/crio-132e364627d8033e5ace8c6ecc9051a899e95febcc1917a5b93af7b11ad4280b WatchSource:0}: Error finding container 132e364627d8033e5ace8c6ecc9051a899e95febcc1917a5b93af7b11ad4280b: Status 404 returned error can't find the container with id 132e364627d8033e5ace8c6ecc9051a899e95febcc1917a5b93af7b11ad4280b
Jul 16 05:00:47.111098 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:47.111058    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd" event={"ID":"f982aded-5030-4aad-b09e-9ecceba4450c","Type":"ContainerStarted","Data":"132e364627d8033e5ace8c6ecc9051a899e95febcc1917a5b93af7b11ad4280b"}
Jul 16 05:00:47.250474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:47.250450    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:47.250751 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:47.250723    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:47.255117 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:47.255099    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:47.270512 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:47.270479    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:48.119478 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:48.119450    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-778d57fdbc-g2hxr"
Jul 16 05:00:48.271573 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:48.271530    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:48.372028 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:48.371941    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:48.472070 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:48.472042    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:48.631935 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:48.631851    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:49.121227 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:49.121196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd" event={"ID":"f982aded-5030-4aad-b09e-9ecceba4450c","Type":"ContainerStarted","Data":"ec5e033e369892cef27679e2a3b7acca9a22367ce0c58e9aada3c2225826f6e0"}
Jul 16 05:00:49.121618 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:49.121417    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:00:49.140258 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:49.140215    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd" podStartSLOduration=0.718316361 podStartE2EDuration="3.140203654s" totalImagesPullingTime="2.421887293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:00:49.138650565 +0000 UTC m=+646.059233360" watchObservedRunningTime="2026-07-16 05:00:49.140203654 +0000 UTC m=+646.060786448"
Jul 16 05:00:51.216952 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.216922    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"]
Jul 16 05:00:51.220332 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.220315    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.222716 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.222690    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tqvjf\""
Jul 16 05:00:51.234503 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.234483    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"]
Jul 16 05:00:51.294302 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.294274    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b25db6-37c5-414e-bd9d-fdddb952a6e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.294409 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.294313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmzzb\" (UniqueName: \"kubernetes.io/projected/73b25db6-37c5-414e-bd9d-fdddb952a6e8-kube-api-access-zmzzb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.395050 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.395012    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b25db6-37c5-414e-bd9d-fdddb952a6e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.395050 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.395058    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmzzb\" (UniqueName: \"kubernetes.io/projected/73b25db6-37c5-414e-bd9d-fdddb952a6e8-kube-api-access-zmzzb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.395361 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.395344    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b25db6-37c5-414e-bd9d-fdddb952a6e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.406221 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.406196    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmzzb\" (UniqueName: \"kubernetes.io/projected/73b25db6-37c5-414e-bd9d-fdddb952a6e8-kube-api-access-zmzzb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.534622 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.534562    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:51.654765 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.654727    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:51.689991 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:51.689963    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"]
Jul 16 05:00:51.693538 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:51.693507    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b25db6_37c5_414e_bd9d_fdddb952a6e8.slice/crio-47f49a862fcd153aa34ac5eaef6f4f166e28602c37912863855d3e79bf947db4 WatchSource:0}: Error finding container 47f49a862fcd153aa34ac5eaef6f4f166e28602c37912863855d3e79bf947db4: Status 404 returned error can't find the container with id 47f49a862fcd153aa34ac5eaef6f4f166e28602c37912863855d3e79bf947db4
Jul 16 05:00:52.134843 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:52.134794    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws" event={"ID":"73b25db6-37c5-414e-bd9d-fdddb952a6e8","Type":"ContainerStarted","Data":"47f49a862fcd153aa34ac5eaef6f4f166e28602c37912863855d3e79bf947db4"}
Jul 16 05:00:52.245539 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:52.245509    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:52.499120 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:52.499039    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:54.484231 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:54.484197    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:55.065252 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.065220    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"]
Jul 16 05:00:55.069546 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.069518    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:00:55.072558 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.072532    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-f7h9l\""
Jul 16 05:00:55.078593 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.078535    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"]
Jul 16 05:00:55.230765 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.230725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzr72\" (UniqueName: \"kubernetes.io/projected/fd1195bc-6a28-49f4-945c-8c2a070527f7-kube-api-access-dzr72\") pod \"authorino-operator-75cc8d5cb7-5j78p\" (UID: \"fd1195bc-6a28-49f4-945c-8c2a070527f7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:00:55.331707 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.331676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzr72\" (UniqueName: \"kubernetes.io/projected/fd1195bc-6a28-49f4-945c-8c2a070527f7-kube-api-access-dzr72\") pod \"authorino-operator-75cc8d5cb7-5j78p\" (UID: \"fd1195bc-6a28-49f4-945c-8c2a070527f7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:00:55.342065 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.342041    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzr72\" (UniqueName: \"kubernetes.io/projected/fd1195bc-6a28-49f4-945c-8c2a070527f7-kube-api-access-dzr72\") pod \"authorino-operator-75cc8d5cb7-5j78p\" (UID: \"fd1195bc-6a28-49f4-945c-8c2a070527f7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:00:55.382943 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.382910    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:00:55.522612 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.522589    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"]
Jul 16 05:00:55.524564 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:55.524532    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1195bc_6a28_49f4_945c_8c2a070527f7.slice/crio-3854231a50d2e871445382314ffbffb442b0cce57e2823e61e9686de1879edb4 WatchSource:0}: Error finding container 3854231a50d2e871445382314ffbffb442b0cce57e2823e61e9686de1879edb4: Status 404 returned error can't find the container with id 3854231a50d2e871445382314ffbffb442b0cce57e2823e61e9686de1879edb4
Jul 16 05:00:55.803299 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:55.803209    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:56.067750 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:56.067716    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:56.155209 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:56.155175    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p" event={"ID":"fd1195bc-6a28-49f4-945c-8c2a070527f7","Type":"ContainerStarted","Data":"3854231a50d2e871445382314ffbffb442b0cce57e2823e61e9686de1879edb4"}
Jul 16 05:00:57.689213 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:57.689174    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:58.300820 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.300788    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"]
Jul 16 05:00:58.303945 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.303926    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:00:58.306728 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.306711    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pf2vj\""
Jul 16 05:00:58.316801 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.316779    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"]
Jul 16 05:00:58.459161 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.459113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6npbg\" (UniqueName: \"kubernetes.io/projected/e1128522-7f45-451e-88c3-56ee99385794-kube-api-access-6npbg\") pod \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" (UID: \"e1128522-7f45-451e-88c3-56ee99385794\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:00:58.560736 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.560646    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npbg\" (UniqueName: \"kubernetes.io/projected/e1128522-7f45-451e-88c3-56ee99385794-kube-api-access-6npbg\") pod \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" (UID: \"e1128522-7f45-451e-88c3-56ee99385794\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:00:58.570978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.570954    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6npbg\" (UniqueName: \"kubernetes.io/projected/e1128522-7f45-451e-88c3-56ee99385794-kube-api-access-6npbg\") pod \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" (UID: \"e1128522-7f45-451e-88c3-56ee99385794\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:00:58.614895 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.614858    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:00:58.774005 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:58.773975    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"]
Jul 16 05:00:58.775909 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:00:58.775884    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1128522_7f45_451e_88c3_56ee99385794.slice/crio-27fb9442cbd6df42659352163a9e91a27533dee608e773a9266216229cfca8fe WatchSource:0}: Error finding container 27fb9442cbd6df42659352163a9e91a27533dee608e773a9266216229cfca8fe: Status 404 returned error can't find the container with id 27fb9442cbd6df42659352163a9e91a27533dee608e773a9266216229cfca8fe
Jul 16 05:00:59.171647 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.171614    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws" event={"ID":"73b25db6-37c5-414e-bd9d-fdddb952a6e8","Type":"ContainerStarted","Data":"d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732"}
Jul 16 05:00:59.171874 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.171781    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:00:59.173254 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.173227    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p" event={"ID":"fd1195bc-6a28-49f4-945c-8c2a070527f7","Type":"ContainerStarted","Data":"6ea210ddc7d597a48ef2b199f1339c4280cec8da9cc95a1ca95687fe90e79a63"}
Jul 16 05:00:59.173367 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.173351    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:00:59.174354 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.174337    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6" event={"ID":"e1128522-7f45-451e-88c3-56ee99385794","Type":"ContainerStarted","Data":"27fb9442cbd6df42659352163a9e91a27533dee608e773a9266216229cfca8fe"}
Jul 16 05:00:59.197144 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.197101    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws" podStartSLOduration=1.308489553 podStartE2EDuration="8.197087355s" totalImagesPullingTime="6.888597802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:00:59.195177729 +0000 UTC m=+656.115760527" watchObservedRunningTime="2026-07-16 05:00:59.197087355 +0000 UTC m=+656.117670148"
Jul 16 05:00:59.211411 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.211361    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p" podStartSLOduration=1.117301917 podStartE2EDuration="4.211348915s" totalImagesPullingTime="3.094046998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:00:59.21105377 +0000 UTC m=+656.131636567" watchObservedRunningTime="2026-07-16 05:00:59.211348915 +0000 UTC m=+656.131931709"
Jul 16 05:00:59.857727 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.857698    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:00:59.954905 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:00:59.954872    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:00.128353 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:00.128259    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jzjzd"
Jul 16 05:01:01.185569 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.185534    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6" event={"ID":"e1128522-7f45-451e-88c3-56ee99385794","Type":"ContainerStarted","Data":"a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394"}
Jul 16 05:01:01.185963 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.185663    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:01.201671 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.201632    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6" podStartSLOduration=1.6204614130000001 podStartE2EDuration="3.201619403s" totalImagesPullingTime="1.58115799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:01:01.200995634 +0000 UTC m=+658.121578429" watchObservedRunningTime="2026-07-16 05:01:01.201619403 +0000 UTC m=+658.122202196"
Jul 16 05:01:01.407848 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.407800    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"]
Jul 16 05:01:01.411046 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.411027    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.413402 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.413382    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 16 05:01:01.413638 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.413621    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k4g7z\""
Jul 16 05:01:01.413702 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.413621    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 16 05:01:01.420333 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.420309    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"]
Jul 16 05:01:01.587487 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.587450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02dc4740-3233-4887-a86d-11a0539ddc8f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.587487 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.587492    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfw4n\" (UniqueName: \"kubernetes.io/projected/02dc4740-3233-4887-a86d-11a0539ddc8f-kube-api-access-tfw4n\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.587655 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.587590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02dc4740-3233-4887-a86d-11a0539ddc8f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.688055 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.688001    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02dc4740-3233-4887-a86d-11a0539ddc8f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.688224 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.688093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02dc4740-3233-4887-a86d-11a0539ddc8f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.688224 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.688116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfw4n\" (UniqueName: \"kubernetes.io/projected/02dc4740-3233-4887-a86d-11a0539ddc8f-kube-api-access-tfw4n\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.688224 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:01.688203    2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 16 05:01:01.688355 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:01.688267    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02dc4740-3233-4887-a86d-11a0539ddc8f-plugin-serving-cert podName:02dc4740-3233-4887-a86d-11a0539ddc8f nodeName:}" failed. No retries permitted until 2026-07-16 05:01:02.188250737 +0000 UTC m=+659.108833531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/02dc4740-3233-4887-a86d-11a0539ddc8f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-nk4k9" (UID: "02dc4740-3233-4887-a86d-11a0539ddc8f") : secret "plugin-serving-cert" not found
Jul 16 05:01:01.688694 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.688676    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02dc4740-3233-4887-a86d-11a0539ddc8f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:01.697315 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:01.697297    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfw4n\" (UniqueName: \"kubernetes.io/projected/02dc4740-3233-4887-a86d-11a0539ddc8f-kube-api-access-tfw4n\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:02.192974 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:02.192941    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02dc4740-3233-4887-a86d-11a0539ddc8f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:02.195383 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:02.195359    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/02dc4740-3233-4887-a86d-11a0539ddc8f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nk4k9\" (UID: \"02dc4740-3233-4887-a86d-11a0539ddc8f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:02.337851 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:02.337804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"
Jul 16 05:01:02.471372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:02.471301    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9"]
Jul 16 05:01:02.474438 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:01:02.474413    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02dc4740_3233_4887_a86d_11a0539ddc8f.slice/crio-73e6a79beda889cb7249a376f8e25b5d7e90b6aed4f5731e7281596d0bbb7bb5 WatchSource:0}: Error finding container 73e6a79beda889cb7249a376f8e25b5d7e90b6aed4f5731e7281596d0bbb7bb5: Status 404 returned error can't find the container with id 73e6a79beda889cb7249a376f8e25b5d7e90b6aed4f5731e7281596d0bbb7bb5
Jul 16 05:01:03.196138 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:03.196103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9" event={"ID":"02dc4740-3233-4887-a86d-11a0539ddc8f","Type":"ContainerStarted","Data":"73e6a79beda889cb7249a376f8e25b5d7e90b6aed4f5731e7281596d0bbb7bb5"}
Jul 16 05:01:03.307583 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:03.307555    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:03.408460 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:03.408425    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:03.648160 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:03.648122    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:03.719347 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:03.719320    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:10.183150 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:10.183112    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:10.183591 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:10.183312    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j78p"
Jul 16 05:01:11.875272 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:11.875236    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"]
Jul 16 05:01:11.880215 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:11.880196    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:11.891285 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:11.891262    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"]
Jul 16 05:01:11.981579 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:11.981539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2b52-3e28-48b4-824b-d82aaf05198d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:11.981741 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:11.981649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrd6\" (UniqueName: \"kubernetes.io/projected/e8fe2b52-3e28-48b4-824b-d82aaf05198d-kube-api-access-rtrd6\") pod \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:11.990714 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:11.990685    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"]
Jul 16 05:01:11.990997 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:11.990969    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-rtrd6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d"
Jul 16 05:01:12.023703 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.023681    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"]
Jul 16 05:01:12.030259 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.030240    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.038222 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.038197    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"]
Jul 16 05:01:12.082252 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.082219    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk8lx\" (UniqueName: \"kubernetes.io/projected/c20445a9-49d5-4485-b571-2a26e1281d77-kube-api-access-mk8lx\") pod \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" (UID: \"c20445a9-49d5-4485-b571-2a26e1281d77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.082411 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.082320    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20445a9-49d5-4485-b571-2a26e1281d77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" (UID: \"c20445a9-49d5-4485-b571-2a26e1281d77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.082411 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.082370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2b52-3e28-48b4-824b-d82aaf05198d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.082517 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.082444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrd6\" (UniqueName: \"kubernetes.io/projected/e8fe2b52-3e28-48b4-824b-d82aaf05198d-kube-api-access-rtrd6\") pod \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.082762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.082738    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2b52-3e28-48b4-824b-d82aaf05198d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.092977 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.092955    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrd6\" (UniqueName: \"kubernetes.io/projected/e8fe2b52-3e28-48b4-824b-d82aaf05198d-kube-api-access-rtrd6\") pod \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.119503 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.119474    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"]
Jul 16 05:01:12.129821 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.129734    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"]
Jul 16 05:01:12.139443 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.139418    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"]
Jul 16 05:01:12.139632 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.139608    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" containerName="manager" containerID="cri-o://d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732" gracePeriod=2
Jul 16 05:01:12.144907 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.144872    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:12.146071 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.145472    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"]
Jul 16 05:01:12.157143 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.157117    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"]
Jul 16 05:01:12.157732 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.157432    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-mk8lx], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb" podUID="c20445a9-49d5-4485-b571-2a26e1281d77"
Jul 16 05:01:12.169686 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.169662    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"]
Jul 16 05:01:12.171526 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.171504    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"]
Jul 16 05:01:12.172029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.172005    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" containerName="manager"
Jul 16 05:01:12.172029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.172029    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" containerName="manager"
Jul 16 05:01:12.172186 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.172127    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" containerName="manager"
Jul 16 05:01:12.175352 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.175326    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.182274 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.182235    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"]
Jul 16 05:01:12.184179 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.184150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk8lx\" (UniqueName: \"kubernetes.io/projected/c20445a9-49d5-4485-b571-2a26e1281d77-kube-api-access-mk8lx\") pod \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" (UID: \"c20445a9-49d5-4485-b571-2a26e1281d77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.184284 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.184203    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20445a9-49d5-4485-b571-2a26e1281d77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" (UID: \"c20445a9-49d5-4485-b571-2a26e1281d77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.184672 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.184648    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20445a9-49d5-4485-b571-2a26e1281d77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" (UID: \"c20445a9-49d5-4485-b571-2a26e1281d77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.186189 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.186167    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"]
Jul 16 05:01:12.186303 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.186273    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.188463 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.188440    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"]
Jul 16 05:01:12.188706 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.188681    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6" podUID="e1128522-7f45-451e-88c3-56ee99385794" containerName="manager" containerID="cri-o://a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394" gracePeriod=2
Jul 16 05:01:12.190219 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.190196    2565 projected.go:196] Error preparing data for projected volume kube-api-access-mk8lx for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-6tpxb" not found
Jul 16 05:01:12.190315 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.190270    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c20445a9-49d5-4485-b571-2a26e1281d77-kube-api-access-mk8lx podName:c20445a9-49d5-4485-b571-2a26e1281d77 nodeName:}" failed. No retries permitted until 2026-07-16 05:01:12.690249957 +0000 UTC m=+669.610832731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mk8lx" (UniqueName: "kubernetes.io/projected/c20445a9-49d5-4485-b571-2a26e1281d77-kube-api-access-mk8lx") pod "kuadrant-operator-controller-manager-86878c769f-6tpxb" (UID: "c20445a9-49d5-4485-b571-2a26e1281d77") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-6tpxb" not found
Jul 16 05:01:12.190644 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.190624    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:12.197709 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.197639    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"]
Jul 16 05:01:12.203371 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.203012    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"]
Jul 16 05:01:12.221235 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.221069    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:12.222067 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.221381    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"]
Jul 16 05:01:12.222067 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.221909    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1128522-7f45-451e-88c3-56ee99385794" containerName="manager"
Jul 16 05:01:12.222067 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.221926    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1128522-7f45-451e-88c3-56ee99385794" containerName="manager"
Jul 16 05:01:12.223112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.222038    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1128522-7f45-451e-88c3-56ee99385794" containerName="manager"
Jul 16 05:01:12.223586 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.223546    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:12.225590 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.225570    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:12.227868 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.227820    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:12.237240 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.237211    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"]
Jul 16 05:01:12.247601 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.247523    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.247601 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.247551    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.253681 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.253651    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:12.255488 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.255456    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:12.257048 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.257023    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.258811 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.258689    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.283677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.283656    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.284789 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.284759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dndgs\" (UniqueName: \"kubernetes.io/projected/9245b950-8611-4e97-96dc-669864bf2e36-kube-api-access-dndgs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d8qp7\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.284916 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.284796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc14451d-9aa2-4b3c-97f4-343d15e2500c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-99j2x\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.284916 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.284889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9245b950-8611-4e97-96dc-669864bf2e36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d8qp7\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.285015 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.284942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdwt9\" (UniqueName: \"kubernetes.io/projected/f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34-kube-api-access-pdwt9\") pod \"limitador-operator-controller-manager-859d7b78bd-2dd9r\" (UID: \"f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:12.285015 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.284976    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqvfc\" (UniqueName: \"kubernetes.io/projected/cc14451d-9aa2-4b3c-97f4-343d15e2500c-kube-api-access-cqvfc\") pod \"kuadrant-operator-controller-manager-86878c769f-99j2x\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.286029 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.285995    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:12.287157 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.287141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.288052 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.288030    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:12.289807 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.289785    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.291565 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.291542    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.293260 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.293228    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:12.294801 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.294776    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:12.297024 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.296987    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:12.298784 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:12.298756    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:12.386031 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.385942    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20445a9-49d5-4485-b571-2a26e1281d77-extensions-socket-volume\") pod \"c20445a9-49d5-4485-b571-2a26e1281d77\" (UID: \"c20445a9-49d5-4485-b571-2a26e1281d77\") "
Jul 16 05:01:12.386031 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386019    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtrd6\" (UniqueName: \"kubernetes.io/projected/e8fe2b52-3e28-48b4-824b-d82aaf05198d-kube-api-access-rtrd6\") pod \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") "
Jul 16 05:01:12.386242 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386061    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2b52-3e28-48b4-824b-d82aaf05198d-extensions-socket-volume\") pod \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\" (UID: \"e8fe2b52-3e28-48b4-824b-d82aaf05198d\") "
Jul 16 05:01:12.386299 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386237    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20445a9-49d5-4485-b571-2a26e1281d77-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c20445a9-49d5-4485-b571-2a26e1281d77" (UID: "c20445a9-49d5-4485-b571-2a26e1281d77"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 05:01:12.386299 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386282    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dndgs\" (UniqueName: \"kubernetes.io/projected/9245b950-8611-4e97-96dc-669864bf2e36-kube-api-access-dndgs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d8qp7\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.386403 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc14451d-9aa2-4b3c-97f4-343d15e2500c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-99j2x\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.386403 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9245b950-8611-4e97-96dc-669864bf2e36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d8qp7\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.386513 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdwt9\" (UniqueName: \"kubernetes.io/projected/f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34-kube-api-access-pdwt9\") pod \"limitador-operator-controller-manager-859d7b78bd-2dd9r\" (UID: \"f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:12.386513 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqvfc\" (UniqueName: \"kubernetes.io/projected/cc14451d-9aa2-4b3c-97f4-343d15e2500c-kube-api-access-cqvfc\") pod \"kuadrant-operator-controller-manager-86878c769f-99j2x\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.386513 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386478    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20445a9-49d5-4485-b571-2a26e1281d77-extensions-socket-volume\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:12.386513 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386479    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fe2b52-3e28-48b4-824b-d82aaf05198d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e8fe2b52-3e28-48b4-824b-d82aaf05198d" (UID: "e8fe2b52-3e28-48b4-824b-d82aaf05198d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 05:01:12.386513 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386490    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk8lx\" (UniqueName: \"kubernetes.io/projected/c20445a9-49d5-4485-b571-2a26e1281d77-kube-api-access-mk8lx\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:12.386989 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386758    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9245b950-8611-4e97-96dc-669864bf2e36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d8qp7\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.386989 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.386765    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc14451d-9aa2-4b3c-97f4-343d15e2500c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-99j2x\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.388609 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.388585    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fe2b52-3e28-48b4-824b-d82aaf05198d-kube-api-access-rtrd6" (OuterVolumeSpecName: "kube-api-access-rtrd6") pod "e8fe2b52-3e28-48b4-824b-d82aaf05198d" (UID: "e8fe2b52-3e28-48b4-824b-d82aaf05198d"). InnerVolumeSpecName "kube-api-access-rtrd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:01:12.399029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.398999    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dndgs\" (UniqueName: \"kubernetes.io/projected/9245b950-8611-4e97-96dc-669864bf2e36-kube-api-access-dndgs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d8qp7\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.399519 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.399471    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqvfc\" (UniqueName: \"kubernetes.io/projected/cc14451d-9aa2-4b3c-97f4-343d15e2500c-kube-api-access-cqvfc\") pod \"kuadrant-operator-controller-manager-86878c769f-99j2x\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.399519 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.399513    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdwt9\" (UniqueName: \"kubernetes.io/projected/f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34-kube-api-access-pdwt9\") pod \"limitador-operator-controller-manager-859d7b78bd-2dd9r\" (UID: \"f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:12.487420 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.487387    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2b52-3e28-48b4-824b-d82aaf05198d-extensions-socket-volume\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:12.487420 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.487422    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtrd6\" (UniqueName: \"kubernetes.io/projected/e8fe2b52-3e28-48b4-824b-d82aaf05198d-kube-api-access-rtrd6\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:12.560510 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.560474    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:12.570131 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.570107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:12.580116 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.580092    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:12.876503 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.876456    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:12.976527 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:12.976479    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:13.252977 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:13.252898    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:13.253141 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:13.252899    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:13.255953 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.255928    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:13.258092 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.258067    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:13.260185 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.260159    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:13.262637 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.262502    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:13.264886 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.264755    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:13.267686 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.267658    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:13.270405 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.270382    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:13.272746 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.272705    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:13.274988 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.274964    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:13.277561 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.277538    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:13.285589 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.285560    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:13.287784 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.287763    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:13.649418 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.649378    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:13.651093 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.651052    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-8sjrd\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-8sjrd"
Jul 16 05:01:13.652665 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.652636    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6tpxb\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6tpxb"
Jul 16 05:01:13.652902 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:13.652885    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c20445a9-49d5-4485-b571-2a26e1281d77" path="/var/lib/kubelet/pods/c20445a9-49d5-4485-b571-2a26e1281d77/volumes"
Jul 16 05:01:13.653197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:13.653182    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8fe2b52-3e28-48b4-824b-d82aaf05198d" path="/var/lib/kubelet/pods/e8fe2b52-3e28-48b4-824b-d82aaf05198d/volumes"
Jul 16 05:01:13.654192 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:13.654170    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:15.867442 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:15.867405    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:15.967815 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:15.967778    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:18.478418 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:18.478380    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:18.580146 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:18.580113    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:22.080430 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:22.080348    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:22.141108 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:22.141062    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:25.680280 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.679677    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:25.682556 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:25.682441    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:25.697194 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.697171    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:25.699228 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:25.699198    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:25.701191 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:25.701163    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:25.707139 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.707116    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"]
Jul 16 05:01:25.708472 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:01:25.708446    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9245b950_8611_4e97_96dc_669864bf2e36.slice/crio-1cce88f9e5949ad0acf765e8f18f8ad48b1faa118e68b76ef280c24d0c86f55e WatchSource:0}: Error finding container 1cce88f9e5949ad0acf765e8f18f8ad48b1faa118e68b76ef280c24d0c86f55e: Status 404 returned error can't find the container with id 1cce88f9e5949ad0acf765e8f18f8ad48b1faa118e68b76ef280c24d0c86f55e
Jul 16 05:01:25.812161 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.812134    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b25db6-37c5-414e-bd9d-fdddb952a6e8-extensions-socket-volume\") pod \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") "
Jul 16 05:01:25.812318 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.812218    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmzzb\" (UniqueName: \"kubernetes.io/projected/73b25db6-37c5-414e-bd9d-fdddb952a6e8-kube-api-access-zmzzb\") pod \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\" (UID: \"73b25db6-37c5-414e-bd9d-fdddb952a6e8\") "
Jul 16 05:01:25.812318 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.812241    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6npbg\" (UniqueName: \"kubernetes.io/projected/e1128522-7f45-451e-88c3-56ee99385794-kube-api-access-6npbg\") pod \"e1128522-7f45-451e-88c3-56ee99385794\" (UID: \"e1128522-7f45-451e-88c3-56ee99385794\") "
Jul 16 05:01:25.812513 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.812489    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73b25db6-37c5-414e-bd9d-fdddb952a6e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "73b25db6-37c5-414e-bd9d-fdddb952a6e8" (UID: "73b25db6-37c5-414e-bd9d-fdddb952a6e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 05:01:25.814233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.814214    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1128522-7f45-451e-88c3-56ee99385794-kube-api-access-6npbg" (OuterVolumeSpecName: "kube-api-access-6npbg") pod "e1128522-7f45-451e-88c3-56ee99385794" (UID: "e1128522-7f45-451e-88c3-56ee99385794"). InnerVolumeSpecName "kube-api-access-6npbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:01:25.814443 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.814423    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b25db6-37c5-414e-bd9d-fdddb952a6e8-kube-api-access-zmzzb" (OuterVolumeSpecName: "kube-api-access-zmzzb") pod "73b25db6-37c5-414e-bd9d-fdddb952a6e8" (UID: "73b25db6-37c5-414e-bd9d-fdddb952a6e8"). InnerVolumeSpecName "kube-api-access-zmzzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:01:25.913579 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.913548    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b25db6-37c5-414e-bd9d-fdddb952a6e8-extensions-socket-volume\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:25.913579 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.913577    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmzzb\" (UniqueName: \"kubernetes.io/projected/73b25db6-37c5-414e-bd9d-fdddb952a6e8-kube-api-access-zmzzb\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:25.913759 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.913589    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6npbg\" (UniqueName: \"kubernetes.io/projected/e1128522-7f45-451e-88c3-56ee99385794-kube-api-access-6npbg\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:25.931361 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.931335    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"]
Jul 16 05:01:25.932860 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:25.932818    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"]
Jul 16 05:01:25.933266 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:01:25.933245    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc14451d_9aa2_4b3c_97f4_343d15e2500c.slice/crio-81c836da8cbc9f1674d51650b897b78dd3197d0acdf537a2a3b75fc491006c04 WatchSource:0}: Error finding container 81c836da8cbc9f1674d51650b897b78dd3197d0acdf537a2a3b75fc491006c04: Status 404 returned error can't find the container with id 81c836da8cbc9f1674d51650b897b78dd3197d0acdf537a2a3b75fc491006c04
Jul 16 05:01:25.933919 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:01:25.933898    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38682a4_db2a_4c8e_b05e_1cf4a0c7dd34.slice/crio-411e07f8902d2890713808e5584d4762574a5f5e9e3f577e9f00af36a845261e WatchSource:0}: Error finding container 411e07f8902d2890713808e5584d4762574a5f5e9e3f577e9f00af36a845261e: Status 404 returned error can't find the container with id 411e07f8902d2890713808e5584d4762574a5f5e9e3f577e9f00af36a845261e
Jul 16 05:01:26.325300 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.325257    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7" event={"ID":"9245b950-8611-4e97-96dc-669864bf2e36","Type":"ContainerStarted","Data":"2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d"}
Jul 16 05:01:26.325488 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.325407    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:26.325488 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.325442    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7" event={"ID":"9245b950-8611-4e97-96dc-669864bf2e36","Type":"ContainerStarted","Data":"1cce88f9e5949ad0acf765e8f18f8ad48b1faa118e68b76ef280c24d0c86f55e"}
Jul 16 05:01:26.326563 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.326541    2565 generic.go:356] "Generic (PLEG): container finished" podID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" containerID="d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732" exitCode=0
Jul 16 05:01:26.326691 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.326574    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:26.326691 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.326634    2565 scope.go:117] "RemoveContainer" containerID="d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732"
Jul 16 05:01:26.327389 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:26.327362    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:26.328231 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.328202    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9" event={"ID":"02dc4740-3233-4887-a86d-11a0539ddc8f","Type":"ContainerStarted","Data":"f7d5a3d3ed2b60d0d1b2465beca95c3d1f299fda4c8e3673630e651723ab875d"}
Jul 16 05:01:26.329146 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:26.329122    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:26.329459 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.329443    2565 generic.go:356] "Generic (PLEG): container finished" podID="e1128522-7f45-451e-88c3-56ee99385794" containerID="a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394" exitCode=0
Jul 16 05:01:26.329527 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.329486    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:26.331096 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.331045    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x" event={"ID":"cc14451d-9aa2-4b3c-97f4-343d15e2500c","Type":"ContainerStarted","Data":"60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5"}
Jul 16 05:01:26.331096 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.331091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x" event={"ID":"cc14451d-9aa2-4b3c-97f4-343d15e2500c","Type":"ContainerStarted","Data":"81c836da8cbc9f1674d51650b897b78dd3197d0acdf537a2a3b75fc491006c04"}
Jul 16 05:01:26.331267 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.331123    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:26.332961 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.332939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r" event={"ID":"f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34","Type":"ContainerStarted","Data":"eb1fb8bd31bea12a8c9890e08fbba79bacdc3680505efc47b338d4648e8a4b3c"}
Jul 16 05:01:26.333065 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.332982    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r" event={"ID":"f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34","Type":"ContainerStarted","Data":"411e07f8902d2890713808e5584d4762574a5f5e9e3f577e9f00af36a845261e"}
Jul 16 05:01:26.333133 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.333073    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:26.336999 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.336985    2565 scope.go:117] "RemoveContainer" containerID="d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732"
Jul 16 05:01:26.337231 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:26.337215    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732\": container with ID starting with d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732 not found: ID does not exist" containerID="d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732"
Jul 16 05:01:26.337277 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.337240    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732"} err="failed to get container status \"d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732\": rpc error: code = NotFound desc = could not find container \"d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732\": container with ID starting with d598adb2437152fc6c681f56ef1ff3406cb9215aaf87ebea41f1867b6fae8732 not found: ID does not exist"
Jul 16 05:01:26.337277 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.337255    2565 scope.go:117] "RemoveContainer" containerID="a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394"
Jul 16 05:01:26.344555 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.344540    2565 scope.go:117] "RemoveContainer" containerID="a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394"
Jul 16 05:01:26.344768 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:26.344752    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394\": container with ID starting with a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394 not found: ID does not exist" containerID="a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394"
Jul 16 05:01:26.344815 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.344775    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394"} err="failed to get container status \"a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394\": rpc error: code = NotFound desc = could not find container \"a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394\": container with ID starting with a6fa741e84646dcb775975c542f9b645fd37806f536b02b13d0842e591f90394 not found: ID does not exist"
Jul 16 05:01:26.349185 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.349149    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7" podStartSLOduration=14.349134821 podStartE2EDuration="14.349134821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:01:26.34735131 +0000 UTC m=+683.267934106" watchObservedRunningTime="2026-07-16 05:01:26.349134821 +0000 UTC m=+683.269717617"
Jul 16 05:01:26.365284 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.365249    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x" podStartSLOduration=14.365238765 podStartE2EDuration="14.365238765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:01:26.363767727 +0000 UTC m=+683.284350520" watchObservedRunningTime="2026-07-16 05:01:26.365238765 +0000 UTC m=+683.285821559"
Jul 16 05:01:26.379020 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.378978    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nk4k9" podStartSLOduration=2.246159715 podStartE2EDuration="25.378963735s" totalImagesPullingTime="23.13280402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:01:26.377769688 +0000 UTC m=+683.298352481" watchObservedRunningTime="2026-07-16 05:01:26.378963735 +0000 UTC m=+683.299546529"
Jul 16 05:01:26.380897 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:26.380868    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2v7t6\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="e1128522-7f45-451e-88c3-56ee99385794" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2v7t6"
Jul 16 05:01:26.382788 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:26.382732    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nsrws\" is forbidden: User \"system:node:ip-10-0-138-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-249.ec2.internal' and this object" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nsrws"
Jul 16 05:01:26.398771 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:26.398735    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r" podStartSLOduration=14.3987225 podStartE2EDuration="14.3987225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:01:26.397332293 +0000 UTC m=+683.317915086" watchObservedRunningTime="2026-07-16 05:01:26.3987225 +0000 UTC m=+683.319305360"
Jul 16 05:01:27.651011 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:27.650983    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73b25db6-37c5-414e-bd9d-fdddb952a6e8" path="/var/lib/kubelet/pods/73b25db6-37c5-414e-bd9d-fdddb952a6e8/volumes"
Jul 16 05:01:27.651466 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:27.651391    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1128522-7f45-451e-88c3-56ee99385794" path="/var/lib/kubelet/pods/e1128522-7f45-451e-88c3-56ee99385794/volumes"
Jul 16 05:01:31.286605 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:31.286571    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:31.387176 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:31.387131    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:34.288007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:34.287974    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:34.392241 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:34.392209    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:37.341918 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.341887    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:01:37.342279 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.341965    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:37.342279 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.342135    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2dd9r"
Jul 16 05:01:37.415398 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.415367    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"]
Jul 16 05:01:37.415581 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.415557    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7" podUID="9245b950-8611-4e97-96dc-669864bf2e36" containerName="manager" containerID="cri-o://2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d" gracePeriod=10
Jul 16 05:01:37.660229 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.660208    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:37.814372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.814345    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9245b950-8611-4e97-96dc-669864bf2e36-extensions-socket-volume\") pod \"9245b950-8611-4e97-96dc-669864bf2e36\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") "
Jul 16 05:01:37.814543 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.814399    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dndgs\" (UniqueName: \"kubernetes.io/projected/9245b950-8611-4e97-96dc-669864bf2e36-kube-api-access-dndgs\") pod \"9245b950-8611-4e97-96dc-669864bf2e36\" (UID: \"9245b950-8611-4e97-96dc-669864bf2e36\") "
Jul 16 05:01:37.814776 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.814753    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9245b950-8611-4e97-96dc-669864bf2e36-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9245b950-8611-4e97-96dc-669864bf2e36" (UID: "9245b950-8611-4e97-96dc-669864bf2e36"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 05:01:37.816460 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.816440    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9245b950-8611-4e97-96dc-669864bf2e36-kube-api-access-dndgs" (OuterVolumeSpecName: "kube-api-access-dndgs") pod "9245b950-8611-4e97-96dc-669864bf2e36" (UID: "9245b950-8611-4e97-96dc-669864bf2e36"). InnerVolumeSpecName "kube-api-access-dndgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:01:37.916014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.915959    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9245b950-8611-4e97-96dc-669864bf2e36-extensions-socket-volume\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:37.916014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:37.915981    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dndgs\" (UniqueName: \"kubernetes.io/projected/9245b950-8611-4e97-96dc-669864bf2e36-kube-api-access-dndgs\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:01:38.391267 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.391234    2565 generic.go:356] "Generic (PLEG): container finished" podID="9245b950-8611-4e97-96dc-669864bf2e36" containerID="2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d" exitCode=0
Jul 16 05:01:38.391668 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.391277    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"
Jul 16 05:01:38.391668 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.391288    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7" event={"ID":"9245b950-8611-4e97-96dc-669864bf2e36","Type":"ContainerDied","Data":"2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d"}
Jul 16 05:01:38.391668 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.391331    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7" event={"ID":"9245b950-8611-4e97-96dc-669864bf2e36","Type":"ContainerDied","Data":"1cce88f9e5949ad0acf765e8f18f8ad48b1faa118e68b76ef280c24d0c86f55e"}
Jul 16 05:01:38.391668 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.391353    2565 scope.go:117] "RemoveContainer" containerID="2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d"
Jul 16 05:01:38.400674 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.400655    2565 scope.go:117] "RemoveContainer" containerID="2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d"
Jul 16 05:01:38.400979 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:01:38.400960    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d\": container with ID starting with 2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d not found: ID does not exist" containerID="2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d"
Jul 16 05:01:38.401024 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.400990    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d"} err="failed to get container status \"2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d\": rpc error: code = NotFound desc = could not find container \"2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d\": container with ID starting with 2c66861624d0a38e70365cd32885c9602c62e15379f68b7914f66ac8e55b118d not found: ID does not exist"
Jul 16 05:01:38.417548 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.417516    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"]
Jul 16 05:01:38.423814 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:38.423791    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d8qp7"]
Jul 16 05:01:39.655440 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:39.655408    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9245b950-8611-4e97-96dc-669864bf2e36" path="/var/lib/kubelet/pods/9245b950-8611-4e97-96dc-669864bf2e36/volumes"
Jul 16 05:01:43.879598 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:43.879567    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:43.979439 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:43.979405    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:53.949446 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:53.949411    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:54.137672 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:54.137634    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:55.693368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.693292    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:55.700132 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.700108    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:01:55.700458 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.700445    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9245b950-8611-4e97-96dc-669864bf2e36" containerName="manager"
Jul 16 05:01:55.700502 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.700459    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9245b950-8611-4e97-96dc-669864bf2e36" containerName="manager"
Jul 16 05:01:55.700535 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.700528    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9245b950-8611-4e97-96dc-669864bf2e36" containerName="manager"
Jul 16 05:01:55.822136 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.822099    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:55.875674 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.875648    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:55.901067 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.901041    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:01:55.901216 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.901076    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:01:55.901216 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.901161    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:55.903576 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.903555    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 05:01:55.960628 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.960570    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6tdq\" (UniqueName: \"kubernetes.io/projected/153d8697-a441-43ba-917c-9fe1b0453ed4-kube-api-access-p6tdq\") pod \"limitador-limitador-64c94fd64f-8mzkp\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:55.960733 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:55.960626    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d8697-a441-43ba-917c-9fe1b0453ed4-config-file\") pod \"limitador-limitador-64c94fd64f-8mzkp\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:56.061591 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.061564    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d8697-a441-43ba-917c-9fe1b0453ed4-config-file\") pod \"limitador-limitador-64c94fd64f-8mzkp\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:56.061710 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.061620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6tdq\" (UniqueName: \"kubernetes.io/projected/153d8697-a441-43ba-917c-9fe1b0453ed4-kube-api-access-p6tdq\") pod \"limitador-limitador-64c94fd64f-8mzkp\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:56.062155 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.062138    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d8697-a441-43ba-917c-9fe1b0453ed4-config-file\") pod \"limitador-limitador-64c94fd64f-8mzkp\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:56.069122 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.069099    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6tdq\" (UniqueName: \"kubernetes.io/projected/153d8697-a441-43ba-917c-9fe1b0453ed4-kube-api-access-p6tdq\") pod \"limitador-limitador-64c94fd64f-8mzkp\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:56.211734 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.211662    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:01:56.398116 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.398088    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:01:56.400332 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:01:56.400309    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod153d8697_a441_43ba_917c_9fe1b0453ed4.slice/crio-d9b7a38a1c93f547eaf2db41223d536e73c146ef51ee8884241edb8167597f7b WatchSource:0}: Error finding container d9b7a38a1c93f547eaf2db41223d536e73c146ef51ee8884241edb8167597f7b: Status 404 returned error can't find the container with id d9b7a38a1c93f547eaf2db41223d536e73c146ef51ee8884241edb8167597f7b
Jul 16 05:01:56.479173 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.479103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp" event={"ID":"153d8697-a441-43ba-917c-9fe1b0453ed4","Type":"ContainerStarted","Data":"d9b7a38a1c93f547eaf2db41223d536e73c146ef51ee8884241edb8167597f7b"}
Jul 16 05:01:56.527165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.527143    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:56.622999 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:56.622969    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:01:58.040326 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:01:58.040269    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:00.502565 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:00.502532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp" event={"ID":"153d8697-a441-43ba-917c-9fe1b0453ed4","Type":"ContainerStarted","Data":"55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a"}
Jul 16 05:02:00.503049 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:00.502638    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:02:00.519559 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:00.519517    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp" podStartSLOduration=2.216946662 podStartE2EDuration="5.519504681s" totalImagesPullingTime="3.302558019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:02:00.517563802 +0000 UTC m=+717.438146597" watchObservedRunningTime="2026-07-16 05:02:00.519504681 +0000 UTC m=+717.440087474"
Jul 16 05:02:09.007251 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:09.007212    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:11.507560 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:11.507524    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:02:13.519850 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:13.519804    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:02:13.520257 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:13.519993    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp" podUID="153d8697-a441-43ba-917c-9fe1b0453ed4" containerName="limitador" containerID="cri-o://55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a" gracePeriod=30
Jul 16 05:02:14.064949 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.064927    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:02:14.116759 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.116683    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6tdq\" (UniqueName: \"kubernetes.io/projected/153d8697-a441-43ba-917c-9fe1b0453ed4-kube-api-access-p6tdq\") pod \"153d8697-a441-43ba-917c-9fe1b0453ed4\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") "
Jul 16 05:02:14.116927 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.116857    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d8697-a441-43ba-917c-9fe1b0453ed4-config-file\") pod \"153d8697-a441-43ba-917c-9fe1b0453ed4\" (UID: \"153d8697-a441-43ba-917c-9fe1b0453ed4\") "
Jul 16 05:02:14.117205 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.117184    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/153d8697-a441-43ba-917c-9fe1b0453ed4-config-file" (OuterVolumeSpecName: "config-file") pod "153d8697-a441-43ba-917c-9fe1b0453ed4" (UID: "153d8697-a441-43ba-917c-9fe1b0453ed4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:02:14.118776 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.118755    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/153d8697-a441-43ba-917c-9fe1b0453ed4-kube-api-access-p6tdq" (OuterVolumeSpecName: "kube-api-access-p6tdq") pod "153d8697-a441-43ba-917c-9fe1b0453ed4" (UID: "153d8697-a441-43ba-917c-9fe1b0453ed4"). InnerVolumeSpecName "kube-api-access-p6tdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:02:14.218231 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.218204    2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d8697-a441-43ba-917c-9fe1b0453ed4-config-file\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:02:14.218231 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.218227    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6tdq\" (UniqueName: \"kubernetes.io/projected/153d8697-a441-43ba-917c-9fe1b0453ed4-kube-api-access-p6tdq\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:02:14.567944 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.567912    2565 generic.go:356] "Generic (PLEG): container finished" podID="153d8697-a441-43ba-917c-9fe1b0453ed4" containerID="55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a" exitCode=0
Jul 16 05:02:14.568387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.567952    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"
Jul 16 05:02:14.568387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.568002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp" event={"ID":"153d8697-a441-43ba-917c-9fe1b0453ed4","Type":"ContainerDied","Data":"55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a"}
Jul 16 05:02:14.568387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.568048    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8mzkp" event={"ID":"153d8697-a441-43ba-917c-9fe1b0453ed4","Type":"ContainerDied","Data":"d9b7a38a1c93f547eaf2db41223d536e73c146ef51ee8884241edb8167597f7b"}
Jul 16 05:02:14.568387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.568069    2565 scope.go:117] "RemoveContainer" containerID="55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a"
Jul 16 05:02:14.576918 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.576900    2565 scope.go:117] "RemoveContainer" containerID="55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a"
Jul 16 05:02:14.577157 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:02:14.577140    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a\": container with ID starting with 55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a not found: ID does not exist" containerID="55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a"
Jul 16 05:02:14.577222 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.577169    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a"} err="failed to get container status \"55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a\": rpc error: code = NotFound desc = could not find container \"55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a\": container with ID starting with 55a28e77e453c5bbc10f3bc9e1d9deeb9b172792d0328816abc52f252a19dc9a not found: ID does not exist"
Jul 16 05:02:14.587321 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.587298    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:02:14.588874 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:14.588857    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8mzkp"]
Jul 16 05:02:15.650881 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:15.650847    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="153d8697-a441-43ba-917c-9fe1b0453ed4" path="/var/lib/kubelet/pods/153d8697-a441-43ba-917c-9fe1b0453ed4/volumes"
Jul 16 05:02:17.127970 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.127930    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:17.220075 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.220042    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:17.598439 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.598412    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"]
Jul 16 05:02:17.598766 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.598753    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="153d8697-a441-43ba-917c-9fe1b0453ed4" containerName="limitador"
Jul 16 05:02:17.598805 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.598768    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="153d8697-a441-43ba-917c-9fe1b0453ed4" containerName="limitador"
Jul 16 05:02:17.598853 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.598846    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="153d8697-a441-43ba-917c-9fe1b0453ed4" containerName="limitador"
Jul 16 05:02:17.602947 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.602919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.605335 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.605154    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 05:02:17.605335 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.605271    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hnpsv\""
Jul 16 05:02:17.606054 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.606033    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 05:02:17.606169 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.606056    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 16 05:02:17.609263 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.609242    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"]
Jul 16 05:02:17.629895 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.629871    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:17.646366 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.646341    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gmjm\" (UniqueName: \"kubernetes.io/projected/6357802a-c0dd-4c21-9514-24b4fd4ddcc7-kube-api-access-4gmjm\") pod \"postgres-69fb4594cf-hj6x7\" (UID: \"6357802a-c0dd-4c21-9514-24b4fd4ddcc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.646472 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.646375    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6357802a-c0dd-4c21-9514-24b4fd4ddcc7-data\") pod \"postgres-69fb4594cf-hj6x7\" (UID: \"6357802a-c0dd-4c21-9514-24b4fd4ddcc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.746929 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.746901    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6357802a-c0dd-4c21-9514-24b4fd4ddcc7-data\") pod \"postgres-69fb4594cf-hj6x7\" (UID: \"6357802a-c0dd-4c21-9514-24b4fd4ddcc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.747067 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.746990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gmjm\" (UniqueName: \"kubernetes.io/projected/6357802a-c0dd-4c21-9514-24b4fd4ddcc7-kube-api-access-4gmjm\") pod \"postgres-69fb4594cf-hj6x7\" (UID: \"6357802a-c0dd-4c21-9514-24b4fd4ddcc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.747250 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.747235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6357802a-c0dd-4c21-9514-24b4fd4ddcc7-data\") pod \"postgres-69fb4594cf-hj6x7\" (UID: \"6357802a-c0dd-4c21-9514-24b4fd4ddcc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.754667 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.754640    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gmjm\" (UniqueName: \"kubernetes.io/projected/6357802a-c0dd-4c21-9514-24b4fd4ddcc7-kube-api-access-4gmjm\") pod \"postgres-69fb4594cf-hj6x7\" (UID: \"6357802a-c0dd-4c21-9514-24b4fd4ddcc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:17.913662 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:17.913580    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:18.039338 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:18.039300    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"]
Jul 16 05:02:18.041666 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:02:18.041642    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6357802a_c0dd_4c21_9514_24b4fd4ddcc7.slice/crio-9a5d84d9a3ca59ebf1d880b787da3bd45f6fab94b36edddccaeaf41393725d66 WatchSource:0}: Error finding container 9a5d84d9a3ca59ebf1d880b787da3bd45f6fab94b36edddccaeaf41393725d66: Status 404 returned error can't find the container with id 9a5d84d9a3ca59ebf1d880b787da3bd45f6fab94b36edddccaeaf41393725d66
Jul 16 05:02:18.590029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:18.589998    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7" event={"ID":"6357802a-c0dd-4c21-9514-24b4fd4ddcc7","Type":"ContainerStarted","Data":"9a5d84d9a3ca59ebf1d880b787da3bd45f6fab94b36edddccaeaf41393725d66"}
Jul 16 05:02:23.616943 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:23.616903    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7" event={"ID":"6357802a-c0dd-4c21-9514-24b4fd4ddcc7","Type":"ContainerStarted","Data":"87f9050846cba000f041ae45f1859a6152a9db89025bd4091dffbb3ad6b0b2d1"}
Jul 16 05:02:23.617331 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:23.616974    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:23.632951 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:23.632911    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7" podStartSLOduration=1.963425386 podStartE2EDuration="6.63289759s" totalImagesPullingTime="4.669472204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:02:23.63102007 +0000 UTC m=+740.551602863" watchObservedRunningTime="2026-07-16 05:02:23.63289759 +0000 UTC m=+740.553480449"
Jul 16 05:02:29.651979 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:29.651952    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hj6x7"
Jul 16 05:02:30.514368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.514335    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cfd8845c5-lp48j"]
Jul 16 05:02:30.520449 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.520424    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.522966 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.522942    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 16 05:02:30.523081 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.522950    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jwp4b\""
Jul 16 05:02:30.524497 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.524469    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cfd8845c5-lp48j"]
Jul 16 05:02:30.663980 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.663953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-tls-cert\") pod \"authorino-cfd8845c5-lp48j\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") " pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.664325 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.663988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9zlz\" (UniqueName: \"kubernetes.io/projected/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-kube-api-access-c9zlz\") pod \"authorino-cfd8845c5-lp48j\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") " pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.764789 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.764710    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-tls-cert\") pod \"authorino-cfd8845c5-lp48j\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") " pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.764789 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.764751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9zlz\" (UniqueName: \"kubernetes.io/projected/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-kube-api-access-c9zlz\") pod \"authorino-cfd8845c5-lp48j\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") " pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.767425 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.767399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-tls-cert\") pod \"authorino-cfd8845c5-lp48j\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") " pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.771952 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.771928    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9zlz\" (UniqueName: \"kubernetes.io/projected/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-kube-api-access-c9zlz\") pod \"authorino-cfd8845c5-lp48j\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") " pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.830971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.830950    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:02:30.959250 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:30.959228    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cfd8845c5-lp48j"]
Jul 16 05:02:30.961496 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:02:30.961470    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52fcc34e_82c0_47b4_8a8f_cd1ff3c30ecf.slice/crio-e9cc9dd3ff2d63fb21e5795452dbcec3641423fdb9bfe18d6c714226d43098ee WatchSource:0}: Error finding container e9cc9dd3ff2d63fb21e5795452dbcec3641423fdb9bfe18d6c714226d43098ee: Status 404 returned error can't find the container with id e9cc9dd3ff2d63fb21e5795452dbcec3641423fdb9bfe18d6c714226d43098ee
Jul 16 05:02:31.653119 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:31.653088    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cfd8845c5-lp48j" event={"ID":"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf","Type":"ContainerStarted","Data":"e9cc9dd3ff2d63fb21e5795452dbcec3641423fdb9bfe18d6c714226d43098ee"}
Jul 16 05:02:33.112197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:33.112150    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:33.211352 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:33.211323    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:35.675684 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:35.675647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cfd8845c5-lp48j" event={"ID":"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf","Type":"ContainerStarted","Data":"2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37"}
Jul 16 05:02:35.689710 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:35.689668    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cfd8845c5-lp48j" podStartSLOduration=1.1896803280000001 podStartE2EDuration="5.689653442s" totalImagesPullingTime="4.499973114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:02:35.68933186 +0000 UTC m=+752.609914659" watchObservedRunningTime="2026-07-16 05:02:35.689653442 +0000 UTC m=+752.610236239"
Jul 16 05:02:40.877690 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.877659    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:40.887878 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.887857    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56b75dc79f-t44k4"]
Jul 16 05:02:40.896368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.896352    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:40.899240 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.899220    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 05:02:40.899854 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.899816    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 05:02:40.899962 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.899860    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xj5pw\""
Jul 16 05:02:40.900998 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.900977    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56b75dc79f-t44k4"]
Jul 16 05:02:40.972940 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:40.972906    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:41.056027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.055996    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lskpx\" (UniqueName: \"kubernetes.io/projected/0a7390e6-803c-4005-8f32-32b74c5fe52e-kube-api-access-lskpx\") pod \"maas-controller-56b75dc79f-t44k4\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") " pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.056164 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.056041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7390e6-803c-4005-8f32-32b74c5fe52e-webhook-cert\") pod \"maas-controller-56b75dc79f-t44k4\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") " pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.072600 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.072573    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:41.079567 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.079542    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-cr784"]
Jul 16 05:02:41.083008 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.082994    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.091646 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.091620    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-cr784"]
Jul 16 05:02:41.156914 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.156813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lskpx\" (UniqueName: \"kubernetes.io/projected/0a7390e6-803c-4005-8f32-32b74c5fe52e-kube-api-access-lskpx\") pod \"maas-controller-56b75dc79f-t44k4\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") " pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.156914 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.156902    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7390e6-803c-4005-8f32-32b74c5fe52e-webhook-cert\") pod \"maas-controller-56b75dc79f-t44k4\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") " pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.159308 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.159288    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7390e6-803c-4005-8f32-32b74c5fe52e-webhook-cert\") pod \"maas-controller-56b75dc79f-t44k4\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") " pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.164742 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.164718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lskpx\" (UniqueName: \"kubernetes.io/projected/0a7390e6-803c-4005-8f32-32b74c5fe52e-kube-api-access-lskpx\") pod \"maas-controller-56b75dc79f-t44k4\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") " pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.173218 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.173186    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:41.208244 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.208219    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:41.258636 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.258592    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/34106d33-89d0-4372-badc-df89a87a182d-webhook-cert\") pod \"maas-controller-679f9cdc6b-cr784\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") " pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.259029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.258662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztvmb\" (UniqueName: \"kubernetes.io/projected/34106d33-89d0-4372-badc-df89a87a182d-kube-api-access-ztvmb\") pod \"maas-controller-679f9cdc6b-cr784\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") " pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.332107 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.332084    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56b75dc79f-t44k4"]
Jul 16 05:02:41.334263 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:02:41.334238    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7390e6_803c_4005_8f32_32b74c5fe52e.slice/crio-1d98384480084c5e7c62fd6ffa7e2f7bc86612ede7c7d6887ac86af474bde9a9 WatchSource:0}: Error finding container 1d98384480084c5e7c62fd6ffa7e2f7bc86612ede7c7d6887ac86af474bde9a9: Status 404 returned error can't find the container with id 1d98384480084c5e7c62fd6ffa7e2f7bc86612ede7c7d6887ac86af474bde9a9
Jul 16 05:02:41.359918 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.359897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/34106d33-89d0-4372-badc-df89a87a182d-webhook-cert\") pod \"maas-controller-679f9cdc6b-cr784\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") " pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.360015 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.359937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztvmb\" (UniqueName: \"kubernetes.io/projected/34106d33-89d0-4372-badc-df89a87a182d-kube-api-access-ztvmb\") pod \"maas-controller-679f9cdc6b-cr784\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") " pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.362215 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.362193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/34106d33-89d0-4372-badc-df89a87a182d-webhook-cert\") pod \"maas-controller-679f9cdc6b-cr784\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") " pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.367243 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.367221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztvmb\" (UniqueName: \"kubernetes.io/projected/34106d33-89d0-4372-badc-df89a87a182d-kube-api-access-ztvmb\") pod \"maas-controller-679f9cdc6b-cr784\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") " pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.393129 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.393109    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:41.522974 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.522902    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-cr784"]
Jul 16 05:02:41.525171 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:02:41.525141    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34106d33_89d0_4372_badc_df89a87a182d.slice/crio-e293aba6a9cc4a03ab34bab5f9fa4f84cc964c0c97f17a0718e6958761d71dc7 WatchSource:0}: Error finding container e293aba6a9cc4a03ab34bab5f9fa4f84cc964c0c97f17a0718e6958761d71dc7: Status 404 returned error can't find the container with id e293aba6a9cc4a03ab34bab5f9fa4f84cc964c0c97f17a0718e6958761d71dc7
Jul 16 05:02:41.701781 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.701707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679f9cdc6b-cr784" event={"ID":"34106d33-89d0-4372-badc-df89a87a182d","Type":"ContainerStarted","Data":"e293aba6a9cc4a03ab34bab5f9fa4f84cc964c0c97f17a0718e6958761d71dc7"}
Jul 16 05:02:41.702823 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:41.702794    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56b75dc79f-t44k4" event={"ID":"0a7390e6-803c-4005-8f32-32b74c5fe52e","Type":"ContainerStarted","Data":"1d98384480084c5e7c62fd6ffa7e2f7bc86612ede7c7d6887ac86af474bde9a9"}
Jul 16 05:02:44.261744 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.261715    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:44.362295 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.362256    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:44.720311 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.720212    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56b75dc79f-t44k4" event={"ID":"0a7390e6-803c-4005-8f32-32b74c5fe52e","Type":"ContainerStarted","Data":"cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b"}
Jul 16 05:02:44.720311 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.720279    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:44.721667 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.721643    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679f9cdc6b-cr784" event={"ID":"34106d33-89d0-4372-badc-df89a87a182d","Type":"ContainerStarted","Data":"76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76"}
Jul 16 05:02:44.721780 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.721756    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:44.737653 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.737613    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56b75dc79f-t44k4" podStartSLOduration=2.046528984 podStartE2EDuration="4.737601419s" totalImagesPullingTime="2.691072435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:02:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:02:44.735523628 +0000 UTC m=+761.656106433" watchObservedRunningTime="2026-07-16 05:02:44.737601419 +0000 UTC m=+761.658184215"
Jul 16 05:02:44.753196 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:44.753155    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-679f9cdc6b-cr784" podStartSLOduration=1.263597782 podStartE2EDuration="3.753143757s" totalImagesPullingTime="2.489545975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:02:44.751707212 +0000 UTC m=+761.672290020" watchObservedRunningTime="2026-07-16 05:02:44.753143757 +0000 UTC m=+761.673726551"
Jul 16 05:02:52.664792 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:52.664760    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:02:55.728345 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:55.728317    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:02:55.728750 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:55.728380    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:55.783382 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:55.783350    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56b75dc79f-t44k4"]
Jul 16 05:02:55.783587 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:55.783561    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56b75dc79f-t44k4" podUID="0a7390e6-803c-4005-8f32-32b74c5fe52e" containerName="manager" containerID="cri-o://cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b" gracePeriod=30
Jul 16 05:02:56.021718 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.021696    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:56.091616 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.091574    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7390e6-803c-4005-8f32-32b74c5fe52e-webhook-cert\") pod \"0a7390e6-803c-4005-8f32-32b74c5fe52e\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") "
Jul 16 05:02:56.091766 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.091631    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lskpx\" (UniqueName: \"kubernetes.io/projected/0a7390e6-803c-4005-8f32-32b74c5fe52e-kube-api-access-lskpx\") pod \"0a7390e6-803c-4005-8f32-32b74c5fe52e\" (UID: \"0a7390e6-803c-4005-8f32-32b74c5fe52e\") "
Jul 16 05:02:56.093797 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.093765    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a7390e6-803c-4005-8f32-32b74c5fe52e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0a7390e6-803c-4005-8f32-32b74c5fe52e" (UID: "0a7390e6-803c-4005-8f32-32b74c5fe52e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:02:56.093903 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.093796    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7390e6-803c-4005-8f32-32b74c5fe52e-kube-api-access-lskpx" (OuterVolumeSpecName: "kube-api-access-lskpx") pod "0a7390e6-803c-4005-8f32-32b74c5fe52e" (UID: "0a7390e6-803c-4005-8f32-32b74c5fe52e"). InnerVolumeSpecName "kube-api-access-lskpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:02:56.192442 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.192403    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lskpx\" (UniqueName: \"kubernetes.io/projected/0a7390e6-803c-4005-8f32-32b74c5fe52e-kube-api-access-lskpx\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:02:56.192442 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.192443    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a7390e6-803c-4005-8f32-32b74c5fe52e-webhook-cert\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:02:56.777239 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.777205    2565 generic.go:356] "Generic (PLEG): container finished" podID="0a7390e6-803c-4005-8f32-32b74c5fe52e" containerID="cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b" exitCode=0
Jul 16 05:02:56.777664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.777246    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56b75dc79f-t44k4"
Jul 16 05:02:56.777664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.777272    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56b75dc79f-t44k4" event={"ID":"0a7390e6-803c-4005-8f32-32b74c5fe52e","Type":"ContainerDied","Data":"cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b"}
Jul 16 05:02:56.777664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.777309    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56b75dc79f-t44k4" event={"ID":"0a7390e6-803c-4005-8f32-32b74c5fe52e","Type":"ContainerDied","Data":"1d98384480084c5e7c62fd6ffa7e2f7bc86612ede7c7d6887ac86af474bde9a9"}
Jul 16 05:02:56.777664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.777330    2565 scope.go:117] "RemoveContainer" containerID="cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b"
Jul 16 05:02:56.786188 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.786041    2565 scope.go:117] "RemoveContainer" containerID="cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b"
Jul 16 05:02:56.786308 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:02:56.786288    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b\": container with ID starting with cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b not found: ID does not exist" containerID="cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b"
Jul 16 05:02:56.786368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.786320    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b"} err="failed to get container status \"cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b\": rpc error: code = NotFound desc = could not find container \"cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b\": container with ID starting with cac19b7ee62b8583edc0858668191498caf6e56ff32d6be87d5f82c34b32728b not found: ID does not exist"
Jul 16 05:02:56.796629 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.796607    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56b75dc79f-t44k4"]
Jul 16 05:02:56.800214 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:56.800195    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56b75dc79f-t44k4"]
Jul 16 05:02:57.655233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:02:57.651639    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7390e6-803c-4005-8f32-32b74c5fe52e" path="/var/lib/kubelet/pods/0a7390e6-803c-4005-8f32-32b74c5fe52e/volumes"
Jul 16 05:03:15.926426 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:15.926391    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:16.026150 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.026106    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:16.589867 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.589817    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"]
Jul 16 05:03:16.590236 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.590221    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7390e6-803c-4005-8f32-32b74c5fe52e" containerName="manager"
Jul 16 05:03:16.590284 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.590238    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7390e6-803c-4005-8f32-32b74c5fe52e" containerName="manager"
Jul 16 05:03:16.590325 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.590306    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7390e6-803c-4005-8f32-32b74c5fe52e" containerName="manager"
Jul 16 05:03:16.598729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.598709    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.601385 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.601357    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kq2xh\""
Jul 16 05:03:16.601491 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.601383    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 16 05:03:16.602075 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.602056    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"]
Jul 16 05:03:16.664295 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.664260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04a29112-991f-46c1-9332-a7723814579e-openshift-service-ca\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.664445 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.664301    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9mhd\" (UniqueName: \"kubernetes.io/projected/04a29112-991f-46c1-9332-a7723814579e-kube-api-access-s9mhd\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.664445 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.664368    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04a29112-991f-46c1-9332-a7723814579e-maas-api-tls\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.765526 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.765495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04a29112-991f-46c1-9332-a7723814579e-openshift-service-ca\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.765643 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.765529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9mhd\" (UniqueName: \"kubernetes.io/projected/04a29112-991f-46c1-9332-a7723814579e-kube-api-access-s9mhd\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.765643 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.765573    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04a29112-991f-46c1-9332-a7723814579e-maas-api-tls\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.765761 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:03:16.765686    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 16 05:03:16.765761 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:03:16.765752    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04a29112-991f-46c1-9332-a7723814579e-maas-api-tls podName:04a29112-991f-46c1-9332-a7723814579e nodeName:}" failed. No retries permitted until 2026-07-16 05:03:17.265732862 +0000 UTC m=+794.186315637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04a29112-991f-46c1-9332-a7723814579e-maas-api-tls") pod "maas-api-7bd6fbb858-fs7hb" (UID: "04a29112-991f-46c1-9332-a7723814579e") : secret "maas-api-serving-cert" not found
Jul 16 05:03:16.766088 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.766068    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04a29112-991f-46c1-9332-a7723814579e-openshift-service-ca\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:16.773954 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:16.773938    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9mhd\" (UniqueName: \"kubernetes.io/projected/04a29112-991f-46c1-9332-a7723814579e-kube-api-access-s9mhd\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:17.172248 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.172219    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"]
Jul 16 05:03:17.175701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.175686    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.178360 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.178340    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 16 05:03:17.178460 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.178426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nzx72\""
Jul 16 05:03:17.185537 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.185520    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"]
Jul 16 05:03:17.270230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.270200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04a29112-991f-46c1-9332-a7723814579e-maas-api-tls\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:17.270393 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.270248    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338d255d-0874-4e20-aae2-358094dca4fc-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-wjxsm\" (UID: \"338d255d-0874-4e20-aae2-358094dca4fc\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.270393 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.270373    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p4rd\" (UniqueName: \"kubernetes.io/projected/338d255d-0874-4e20-aae2-358094dca4fc-kube-api-access-6p4rd\") pod \"payload-pre-processing-74584bb9f8-wjxsm\" (UID: \"338d255d-0874-4e20-aae2-358094dca4fc\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.273184 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.273151    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04a29112-991f-46c1-9332-a7723814579e-maas-api-tls\") pod \"maas-api-7bd6fbb858-fs7hb\" (UID: \"04a29112-991f-46c1-9332-a7723814579e\") " pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:17.371059 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.371025    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338d255d-0874-4e20-aae2-358094dca4fc-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-wjxsm\" (UID: \"338d255d-0874-4e20-aae2-358094dca4fc\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.371236 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.371071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p4rd\" (UniqueName: \"kubernetes.io/projected/338d255d-0874-4e20-aae2-358094dca4fc-kube-api-access-6p4rd\") pod \"payload-pre-processing-74584bb9f8-wjxsm\" (UID: \"338d255d-0874-4e20-aae2-358094dca4fc\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.371767 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.371747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338d255d-0874-4e20-aae2-358094dca4fc-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-wjxsm\" (UID: \"338d255d-0874-4e20-aae2-358094dca4fc\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.379062 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.379027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p4rd\" (UniqueName: \"kubernetes.io/projected/338d255d-0874-4e20-aae2-358094dca4fc-kube-api-access-6p4rd\") pod \"payload-pre-processing-74584bb9f8-wjxsm\" (UID: \"338d255d-0874-4e20-aae2-358094dca4fc\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.487768 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.487693    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:17.510965 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.510922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:17.665089 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.665061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"]
Jul 16 05:03:17.666811 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:03:17.666786    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a29112_991f_46c1_9332_a7723814579e.slice/crio-81689bad314bf0c7538916374a2b70703009496b805ff440ca15292e30d8db71 WatchSource:0}: Error finding container 81689bad314bf0c7538916374a2b70703009496b805ff440ca15292e30d8db71: Status 404 returned error can't find the container with id 81689bad314bf0c7538916374a2b70703009496b805ff440ca15292e30d8db71
Jul 16 05:03:17.842233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.842207    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"]
Jul 16 05:03:17.844423 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:03:17.844232    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338d255d_0874_4e20_aae2_358094dca4fc.slice/crio-5058b7a27f4fa68803774dbe1409911f95edaf0d7c922c0000a66fd1cdbf1146 WatchSource:0}: Error finding container 5058b7a27f4fa68803774dbe1409911f95edaf0d7c922c0000a66fd1cdbf1146: Status 404 returned error can't find the container with id 5058b7a27f4fa68803774dbe1409911f95edaf0d7c922c0000a66fd1cdbf1146
Jul 16 05:03:17.876795 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.876769    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb" event={"ID":"04a29112-991f-46c1-9332-a7723814579e","Type":"ContainerStarted","Data":"81689bad314bf0c7538916374a2b70703009496b805ff440ca15292e30d8db71"}
Jul 16 05:03:17.877631 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:17.877611    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm" event={"ID":"338d255d-0874-4e20-aae2-358094dca4fc","Type":"ContainerStarted","Data":"5058b7a27f4fa68803774dbe1409911f95edaf0d7c922c0000a66fd1cdbf1146"}
Jul 16 05:03:19.281128 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.281090    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:19.332262 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.332235    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:19.890478 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.890444    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb" event={"ID":"04a29112-991f-46c1-9332-a7723814579e","Type":"ContainerStarted","Data":"40aa7c1fe959c5d6e8b61d93d49733dcd9b6da0c8691edf7949c6a6fda0b1649"}
Jul 16 05:03:19.890668 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.890512    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:19.892226 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.892198    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm" event={"ID":"338d255d-0874-4e20-aae2-358094dca4fc","Type":"ContainerStarted","Data":"178432875f8c2613f4d294da4497950f2e9d601b1e54620ddd61ef4d78fea3f4"}
Jul 16 05:03:19.892364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.892336    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:19.906686 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.906611    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb" podStartSLOduration=2.109533522 podStartE2EDuration="3.906594126s" totalImagesPullingTime="1.797060604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:03:19.90581853 +0000 UTC m=+796.826401325" watchObservedRunningTime="2026-07-16 05:03:19.906594126 +0000 UTC m=+796.827176919"
Jul 16 05:03:19.920843 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:19.920788    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm" podStartSLOduration=1.304728249 podStartE2EDuration="2.92077418s" totalImagesPullingTime="1.616045931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:03:19.920072128 +0000 UTC m=+796.840654922" watchObservedRunningTime="2026-07-16 05:03:19.92077418 +0000 UTC m=+796.841356973"
Jul 16 05:03:25.902609 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:25.902537    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7bd6fbb858-fs7hb"
Jul 16 05:03:27.631812 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:27.631782    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:27.729417 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:27.729388    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:30.899684 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:30.899653    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-wjxsm"
Jul 16 05:03:39.537916 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.537877    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"]
Jul 16 05:03:39.549978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.549951    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.551787 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.551759    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"]
Jul 16 05:03:39.552374 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.552347    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 05:03:39.553405 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.553384    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qtz5t\""
Jul 16 05:03:39.553541 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.553404    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 05:03:39.553541 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.553391    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 05:03:39.571200 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.571177    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:03:39.655320 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.655296    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6192500c-962e-4954-a4a5-d2020c1e219b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.655468 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.655327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.655468 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.655412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.655468 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.655457    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.655670 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.655500    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.655670 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.655602    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smfxd\" (UniqueName: \"kubernetes.io/projected/6192500c-962e-4954-a4a5-d2020c1e219b-kube-api-access-smfxd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756163 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756207    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756272    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smfxd\" (UniqueName: \"kubernetes.io/projected/6192500c-962e-4954-a4a5-d2020c1e219b-kube-api-access-smfxd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756463 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756339    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6192500c-962e-4954-a4a5-d2020c1e219b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756463 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756368    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.756567 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.756524    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.757285 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.757263    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.757745 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.757674    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.759255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.759233    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6192500c-962e-4954-a4a5-d2020c1e219b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.759351 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.759334    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6192500c-962e-4954-a4a5-d2020c1e219b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.764075 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.764057    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smfxd\" (UniqueName: \"kubernetes.io/projected/6192500c-962e-4954-a4a5-d2020c1e219b-kube-api-access-smfxd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc\" (UID: \"6192500c-962e-4954-a4a5-d2020c1e219b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.861198 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.861172    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:39.993040 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:39.993007    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"]
Jul 16 05:03:39.994058 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:03:39.994032    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6192500c_962e_4954_a4a5_d2020c1e219b.slice/crio-90ca2b91559e77ac6831bf85ab37f239073f74e6a386583baaf0ce7c32cdb252 WatchSource:0}: Error finding container 90ca2b91559e77ac6831bf85ab37f239073f74e6a386583baaf0ce7c32cdb252: Status 404 returned error can't find the container with id 90ca2b91559e77ac6831bf85ab37f239073f74e6a386583baaf0ce7c32cdb252
Jul 16 05:03:40.991192 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:40.991155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc" event={"ID":"6192500c-962e-4954-a4a5-d2020c1e219b","Type":"ContainerStarted","Data":"90ca2b91559e77ac6831bf85ab37f239073f74e6a386583baaf0ce7c32cdb252"}
Jul 16 05:03:41.997543 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:41.997509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc" event={"ID":"6192500c-962e-4954-a4a5-d2020c1e219b","Type":"ContainerStarted","Data":"2bed3e7b03c38444d02456779098ad0d7c9f0290f87820999f7b21e5f8b47491"}
Jul 16 05:03:42.013104 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:42.013059    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc" podStartSLOduration=1.8288074029999999 podStartE2EDuration="3.013044384s" totalImagesPullingTime="1.184236981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:03:42.011748329 +0000 UTC m=+818.932331123" watchObservedRunningTime="2026-07-16 05:03:42.013044384 +0000 UTC m=+818.933627201"
Jul 16 05:03:49.861579 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:49.861536    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:49.862174 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:49.861607    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:49.873600 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:49.873578    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:50.043770 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:50.043747    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc"
Jul 16 05:03:52.262088 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:03:52.262049    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:04:04.162743 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:04.162707    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:04:13.337680 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.337642    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"]
Jul 16 05:04:13.344079 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.344056    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.346533 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.346510    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 16 05:04:13.351619 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.351596    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"]
Jul 16 05:04:13.427434 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.427393    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.427620 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.427497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.427620 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.427539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.427620 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.427567    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.427620 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.427582    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.427772 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.427686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v84dr\" (UniqueName: \"kubernetes.io/projected/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-kube-api-access-v84dr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.436187 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.436157    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:04:13.528968 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.528940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529085 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.528970    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529085 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v84dr\" (UniqueName: \"kubernetes.io/projected/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-kube-api-access-v84dr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529085 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529054    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529211 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529119    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529280 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529367 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529348    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529426 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529408    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.529480 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.529468    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.531402 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.531376    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.531723 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.531707    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.538865 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.538845    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v84dr\" (UniqueName: \"kubernetes.io/projected/3ebb260d-90f4-4ab5-9435-53ea9f40b4ef-kube-api-access-v84dr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv\" (UID: \"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.655020 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.654954    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:13.791412 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:13.791389    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"]
Jul 16 05:04:13.793225 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:04:13.793199    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebb260d_90f4_4ab5_9435_53ea9f40b4ef.slice/crio-f1699318ff0d2770ae7e600079fd66b5560aea0177ddfa1faf70b4ebb35c882e WatchSource:0}: Error finding container f1699318ff0d2770ae7e600079fd66b5560aea0177ddfa1faf70b4ebb35c882e: Status 404 returned error can't find the container with id f1699318ff0d2770ae7e600079fd66b5560aea0177ddfa1faf70b4ebb35c882e
Jul 16 05:04:14.148472 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:14.148437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv" event={"ID":"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef","Type":"ContainerStarted","Data":"2958103e48ee383f986c922bedef55e0e4bbdeb5e2bec54375b7bbbb81c699c7"}
Jul 16 05:04:14.148593 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:14.148499    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv" event={"ID":"3ebb260d-90f4-4ab5-9435-53ea9f40b4ef","Type":"ContainerStarted","Data":"f1699318ff0d2770ae7e600079fd66b5560aea0177ddfa1faf70b4ebb35c882e"}
Jul 16 05:04:14.166863 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:14.166796    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv" podStartSLOduration=0.967308744 podStartE2EDuration="1.166781125s" totalImagesPullingTime="199.472381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:04:14.165683796 +0000 UTC m=+851.086266591" watchObservedRunningTime="2026-07-16 05:04:14.166781125 +0000 UTC m=+851.087363918"
Jul 16 05:04:20.647535 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:20.647495    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:04:23.656006 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:23.655964    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:23.656554 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:23.656033    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:23.668068 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:23.668044    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:24.208098 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:24.208066    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv"
Jul 16 05:04:28.639095 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:04:28.639055    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:05:04.228481 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:04.228373    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:05:04.232799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:04.228588    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:05:25.652772 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:25.652737    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cfd8845c5-lp48j"]
Jul 16 05:05:25.653522 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:25.653483    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cfd8845c5-lp48j" podUID="52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" containerName="authorino" containerID="cri-o://2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37" gracePeriod=30
Jul 16 05:05:25.922634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:25.922606    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:05:25.988516 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:25.988487    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-tls-cert\") pod \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") "
Jul 16 05:05:25.988865 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:25.988561    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9zlz\" (UniqueName: \"kubernetes.io/projected/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-kube-api-access-c9zlz\") pod \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\" (UID: \"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf\") "
Jul 16 05:05:25.990781 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:25.990742    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-kube-api-access-c9zlz" (OuterVolumeSpecName: "kube-api-access-c9zlz") pod "52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" (UID: "52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf"). InnerVolumeSpecName "kube-api-access-c9zlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:05:26.000673 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.000642    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" (UID: "52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:05:26.089624 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.089586    2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-tls-cert\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:05:26.089624 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.089618    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9zlz\" (UniqueName: \"kubernetes.io/projected/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf-kube-api-access-c9zlz\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:05:26.492957 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.492921    2565 generic.go:356] "Generic (PLEG): container finished" podID="52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" containerID="2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37" exitCode=0
Jul 16 05:05:26.493158 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.492980    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cfd8845c5-lp48j"
Jul 16 05:05:26.493158 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.493017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cfd8845c5-lp48j" event={"ID":"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf","Type":"ContainerDied","Data":"2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37"}
Jul 16 05:05:26.493158 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.493064    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cfd8845c5-lp48j" event={"ID":"52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf","Type":"ContainerDied","Data":"e9cc9dd3ff2d63fb21e5795452dbcec3641423fdb9bfe18d6c714226d43098ee"}
Jul 16 05:05:26.493158 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.493081    2565 scope.go:117] "RemoveContainer" containerID="2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37"
Jul 16 05:05:26.502230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.502211    2565 scope.go:117] "RemoveContainer" containerID="2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37"
Jul 16 05:05:26.502493 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:05:26.502473    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37\": container with ID starting with 2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37 not found: ID does not exist" containerID="2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37"
Jul 16 05:05:26.502555 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.502508    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37"} err="failed to get container status \"2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37\": rpc error: code = NotFound desc = could not find container \"2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37\": container with ID starting with 2985eaf9c443fc1104003e5647af8b23b71ee773164cf4940b1ba0c82e345b37 not found: ID does not exist"
Jul 16 05:05:26.515980 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.515957    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cfd8845c5-lp48j"]
Jul 16 05:05:26.519781 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:26.519759    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cfd8845c5-lp48j"]
Jul 16 05:05:27.237081 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:27.237045    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:05:27.651659 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:27.651616    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" path="/var/lib/kubelet/pods/52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf/volumes"
Jul 16 05:05:29.510550 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:05:29.510516    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:06:49.898613 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:49.898571    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:06:50.004308 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.004273    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-cr784"]
Jul 16 05:06:50.004515 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.004493    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-679f9cdc6b-cr784" podUID="34106d33-89d0-4372-badc-df89a87a182d" containerName="manager" containerID="cri-o://76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76" gracePeriod=30
Jul 16 05:06:50.257625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.257598    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:06:50.345753 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.345717    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/34106d33-89d0-4372-badc-df89a87a182d-webhook-cert\") pod \"34106d33-89d0-4372-badc-df89a87a182d\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") "
Jul 16 05:06:50.345963 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.345820    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztvmb\" (UniqueName: \"kubernetes.io/projected/34106d33-89d0-4372-badc-df89a87a182d-kube-api-access-ztvmb\") pod \"34106d33-89d0-4372-badc-df89a87a182d\" (UID: \"34106d33-89d0-4372-badc-df89a87a182d\") "
Jul 16 05:06:50.347822 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.347798    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34106d33-89d0-4372-badc-df89a87a182d-kube-api-access-ztvmb" (OuterVolumeSpecName: "kube-api-access-ztvmb") pod "34106d33-89d0-4372-badc-df89a87a182d" (UID: "34106d33-89d0-4372-badc-df89a87a182d"). InnerVolumeSpecName "kube-api-access-ztvmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:06:50.347934 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.347905    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34106d33-89d0-4372-badc-df89a87a182d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "34106d33-89d0-4372-badc-df89a87a182d" (UID: "34106d33-89d0-4372-badc-df89a87a182d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:06:50.446540 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.446483    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/34106d33-89d0-4372-badc-df89a87a182d-webhook-cert\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:06:50.446540 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.446506    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztvmb\" (UniqueName: \"kubernetes.io/projected/34106d33-89d0-4372-badc-df89a87a182d-kube-api-access-ztvmb\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:06:50.904954 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.904920    2565 generic.go:356] "Generic (PLEG): container finished" podID="34106d33-89d0-4372-badc-df89a87a182d" containerID="76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76" exitCode=0
Jul 16 05:06:50.905328 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.904958    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679f9cdc6b-cr784"
Jul 16 05:06:50.905328 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.905004    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679f9cdc6b-cr784" event={"ID":"34106d33-89d0-4372-badc-df89a87a182d","Type":"ContainerDied","Data":"76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76"}
Jul 16 05:06:50.905328 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.905057    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679f9cdc6b-cr784" event={"ID":"34106d33-89d0-4372-badc-df89a87a182d","Type":"ContainerDied","Data":"e293aba6a9cc4a03ab34bab5f9fa4f84cc964c0c97f17a0718e6958761d71dc7"}
Jul 16 05:06:50.905328 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.905086    2565 scope.go:117] "RemoveContainer" containerID="76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76"
Jul 16 05:06:50.914308 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.914290    2565 scope.go:117] "RemoveContainer" containerID="76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76"
Jul 16 05:06:50.914546 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:06:50.914525    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76\": container with ID starting with 76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76 not found: ID does not exist" containerID="76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76"
Jul 16 05:06:50.914596 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.914558    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76"} err="failed to get container status \"76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76\": rpc error: code = NotFound desc = could not find container \"76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76\": container with ID starting with 76c66fb3ec585ae61c6da82fec6d1cf50c62a6489814f75d3f0072acf03afa76 not found: ID does not exist"
Jul 16 05:06:50.925419 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.925392    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-cr784"]
Jul 16 05:06:50.929527 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:50.929503    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-cr784"]
Jul 16 05:06:51.622787 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.622754    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:06:51.651007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.650978    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34106d33-89d0-4372-badc-df89a87a182d" path="/var/lib/kubelet/pods/34106d33-89d0-4372-badc-df89a87a182d/volumes"
Jul 16 05:06:51.772711 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.772681    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-b8zfj"]
Jul 16 05:06:51.773070 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.773058    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" containerName="authorino"
Jul 16 05:06:51.773124 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.773072    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" containerName="authorino"
Jul 16 05:06:51.773124 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.773088    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34106d33-89d0-4372-badc-df89a87a182d" containerName="manager"
Jul 16 05:06:51.773124 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.773094    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="34106d33-89d0-4372-badc-df89a87a182d" containerName="manager"
Jul 16 05:06:51.773230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.773170    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52fcc34e-82c0-47b4-8a8f-cd1ff3c30ecf" containerName="authorino"
Jul 16 05:06:51.773230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.773180    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="34106d33-89d0-4372-badc-df89a87a182d" containerName="manager"
Jul 16 05:06:51.777565 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.777549    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:51.780870 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.780846    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 05:06:51.781006 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.780968    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 05:06:51.781125 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.781074    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xj5pw\""
Jul 16 05:06:51.783941 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.783917    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-b8zfj"]
Jul 16 05:06:51.859802 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.859764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxc95\" (UniqueName: \"kubernetes.io/projected/caf0b2bd-187f-47b5-a4cf-f7056398dbd7-kube-api-access-xxc95\") pod \"maas-controller-679f9cdc6b-b8zfj\" (UID: \"caf0b2bd-187f-47b5-a4cf-f7056398dbd7\") " pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:51.860021 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.859886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf0b2bd-187f-47b5-a4cf-f7056398dbd7-webhook-cert\") pod \"maas-controller-679f9cdc6b-b8zfj\" (UID: \"caf0b2bd-187f-47b5-a4cf-f7056398dbd7\") " pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:51.961054 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.960961    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf0b2bd-187f-47b5-a4cf-f7056398dbd7-webhook-cert\") pod \"maas-controller-679f9cdc6b-b8zfj\" (UID: \"caf0b2bd-187f-47b5-a4cf-f7056398dbd7\") " pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:51.961054 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.961031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxc95\" (UniqueName: \"kubernetes.io/projected/caf0b2bd-187f-47b5-a4cf-f7056398dbd7-kube-api-access-xxc95\") pod \"maas-controller-679f9cdc6b-b8zfj\" (UID: \"caf0b2bd-187f-47b5-a4cf-f7056398dbd7\") " pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:51.963332 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.963306    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf0b2bd-187f-47b5-a4cf-f7056398dbd7-webhook-cert\") pod \"maas-controller-679f9cdc6b-b8zfj\" (UID: \"caf0b2bd-187f-47b5-a4cf-f7056398dbd7\") " pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:51.969148 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:51.969127    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxc95\" (UniqueName: \"kubernetes.io/projected/caf0b2bd-187f-47b5-a4cf-f7056398dbd7-kube-api-access-xxc95\") pod \"maas-controller-679f9cdc6b-b8zfj\" (UID: \"caf0b2bd-187f-47b5-a4cf-f7056398dbd7\") " pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:52.089395 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.089355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:52.226890 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.226862    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-679f9cdc6b-b8zfj"]
Jul 16 05:06:52.228027 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:06:52.227994    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf0b2bd_187f_47b5_a4cf_f7056398dbd7.slice/crio-25072f47fda10562cd21f16f2fdf52feb76543b00c8efa7fe7917b24228f634b WatchSource:0}: Error finding container 25072f47fda10562cd21f16f2fdf52feb76543b00c8efa7fe7917b24228f634b: Status 404 returned error can't find the container with id 25072f47fda10562cd21f16f2fdf52feb76543b00c8efa7fe7917b24228f634b
Jul 16 05:06:52.229592 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.229571    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 05:06:52.917340 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.917297    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj" event={"ID":"caf0b2bd-187f-47b5-a4cf-f7056398dbd7","Type":"ContainerStarted","Data":"c51aa0503455c31b5da454ad6f6ddf6268c030535572f097eb557a1616ceadec"}
Jul 16 05:06:52.917520 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.917367    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj" event={"ID":"caf0b2bd-187f-47b5-a4cf-f7056398dbd7","Type":"ContainerStarted","Data":"25072f47fda10562cd21f16f2fdf52feb76543b00c8efa7fe7917b24228f634b"}
Jul 16 05:06:52.917520 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.917437    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:06:52.932431 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:06:52.932360    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj" podStartSLOduration=1.451974431 podStartE2EDuration="1.932338308s" totalImagesPullingTime="480.363877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:06:52.931144887 +0000 UTC m=+1009.851727680" watchObservedRunningTime="2026-07-16 05:06:52.932338308 +0000 UTC m=+1009.852921100"
Jul 16 05:07:03.924392 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:03.924311    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-679f9cdc6b-b8zfj"
Jul 16 05:07:17.079293 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:17.079245    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:17.182267 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:17.182225    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:34.813414 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:34.813378    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:34.927148 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:34.927117    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:35.086946 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:35.086912    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:40.897410 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:40.897369    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:40.963097 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:40.963063    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:41.060749 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:41.060708    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:41.180174 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:41.180065    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:41.314804 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:41.314760    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:46.955395 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:46.955364    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:47.019817 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:47.019779    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:56.915612 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:56.915529    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:07:56.965638 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:07:56.965604    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:08:08.353120 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:08:08.353088    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:08:08.453141 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:08:08.453111    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:08:40.868751 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:08:40.868721    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:08:40.936226 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:08:40.936193    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:10:04.259054 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:10:04.258935    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:10:04.262183 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:10:04.259383    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:11:02.205717 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:02.205645    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:11:46.134696 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:46.134657    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"]
Jul 16 05:11:46.135230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:46.134864    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x" podUID="cc14451d-9aa2-4b3c-97f4-343d15e2500c" containerName="manager" containerID="cri-o://60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5" gracePeriod=10
Jul 16 05:11:48.679210 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.679181    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:11:48.845285 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.845256    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc14451d-9aa2-4b3c-97f4-343d15e2500c-extensions-socket-volume\") pod \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") "
Jul 16 05:11:48.845429 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.845343    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqvfc\" (UniqueName: \"kubernetes.io/projected/cc14451d-9aa2-4b3c-97f4-343d15e2500c-kube-api-access-cqvfc\") pod \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\" (UID: \"cc14451d-9aa2-4b3c-97f4-343d15e2500c\") "
Jul 16 05:11:48.845671 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.845648    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc14451d-9aa2-4b3c-97f4-343d15e2500c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cc14451d-9aa2-4b3c-97f4-343d15e2500c" (UID: "cc14451d-9aa2-4b3c-97f4-343d15e2500c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 05:11:48.847377 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.847357    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc14451d-9aa2-4b3c-97f4-343d15e2500c-kube-api-access-cqvfc" (OuterVolumeSpecName: "kube-api-access-cqvfc") pod "cc14451d-9aa2-4b3c-97f4-343d15e2500c" (UID: "cc14451d-9aa2-4b3c-97f4-343d15e2500c"). InnerVolumeSpecName "kube-api-access-cqvfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:11:48.946697 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.946668    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc14451d-9aa2-4b3c-97f4-343d15e2500c-extensions-socket-volume\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:11:48.946697 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:48.946695    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqvfc\" (UniqueName: \"kubernetes.io/projected/cc14451d-9aa2-4b3c-97f4-343d15e2500c-kube-api-access-cqvfc\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:11:49.349596 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.349562    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc14451d-9aa2-4b3c-97f4-343d15e2500c" containerID="60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5" exitCode=0
Jul 16 05:11:49.349806 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.349610    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"
Jul 16 05:11:49.349806 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.349659    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x" event={"ID":"cc14451d-9aa2-4b3c-97f4-343d15e2500c","Type":"ContainerDied","Data":"60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5"}
Jul 16 05:11:49.349806 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.349723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x" event={"ID":"cc14451d-9aa2-4b3c-97f4-343d15e2500c","Type":"ContainerDied","Data":"81c836da8cbc9f1674d51650b897b78dd3197d0acdf537a2a3b75fc491006c04"}
Jul 16 05:11:49.349806 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.349799    2565 scope.go:117] "RemoveContainer" containerID="60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5"
Jul 16 05:11:49.360615 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.360604    2565 scope.go:117] "RemoveContainer" containerID="60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5"
Jul 16 05:11:49.360882 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:11:49.360860    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5\": container with ID starting with 60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5 not found: ID does not exist" containerID="60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5"
Jul 16 05:11:49.361004 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.360888    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5"} err="failed to get container status \"60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5\": rpc error: code = NotFound desc = could not find container \"60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5\": container with ID starting with 60f1fac91a04ad57ad02a680f410bdf82a43db48e471357a45b8a98314c134d5 not found: ID does not exist"
Jul 16 05:11:49.370711 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.370689    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"]
Jul 16 05:11:49.374364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.374337    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-99j2x"]
Jul 16 05:11:49.651367 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:49.651282    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc14451d-9aa2-4b3c-97f4-343d15e2500c" path="/var/lib/kubelet/pods/cc14451d-9aa2-4b3c-97f4-343d15e2500c/volumes"
Jul 16 05:11:54.124349 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:11:54.124319    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:12:47.327614 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.327566    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"]
Jul 16 05:12:47.328157 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.328141    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc14451d-9aa2-4b3c-97f4-343d15e2500c" containerName="manager"
Jul 16 05:12:47.328203 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.328159    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc14451d-9aa2-4b3c-97f4-343d15e2500c" containerName="manager"
Jul 16 05:12:47.328281 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.328270    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc14451d-9aa2-4b3c-97f4-343d15e2500c" containerName="manager"
Jul 16 05:12:47.331659 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.331642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.334037 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.334016    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tqvjf\""
Jul 16 05:12:47.340894 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.340876    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"]
Jul 16 05:12:47.450663 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.450628    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f91ccbe1-378f-43f8-ab0d-3d2e961b2890-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4dscs\" (UID: \"f91ccbe1-378f-43f8-ab0d-3d2e961b2890\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.450865 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.450689    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt5fn\" (UniqueName: \"kubernetes.io/projected/f91ccbe1-378f-43f8-ab0d-3d2e961b2890-kube-api-access-kt5fn\") pod \"kuadrant-operator-controller-manager-86878c769f-4dscs\" (UID: \"f91ccbe1-378f-43f8-ab0d-3d2e961b2890\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.551201 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.551164    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f91ccbe1-378f-43f8-ab0d-3d2e961b2890-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4dscs\" (UID: \"f91ccbe1-378f-43f8-ab0d-3d2e961b2890\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.551395 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.551227    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt5fn\" (UniqueName: \"kubernetes.io/projected/f91ccbe1-378f-43f8-ab0d-3d2e961b2890-kube-api-access-kt5fn\") pod \"kuadrant-operator-controller-manager-86878c769f-4dscs\" (UID: \"f91ccbe1-378f-43f8-ab0d-3d2e961b2890\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.551546 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.551522    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f91ccbe1-378f-43f8-ab0d-3d2e961b2890-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4dscs\" (UID: \"f91ccbe1-378f-43f8-ab0d-3d2e961b2890\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.560860 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.560824    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt5fn\" (UniqueName: \"kubernetes.io/projected/f91ccbe1-378f-43f8-ab0d-3d2e961b2890-kube-api-access-kt5fn\") pod \"kuadrant-operator-controller-manager-86878c769f-4dscs\" (UID: \"f91ccbe1-378f-43f8-ab0d-3d2e961b2890\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.643120 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.643066    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:47.777807 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:47.777777    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"]
Jul 16 05:12:47.779378 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:12:47.779353    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91ccbe1_378f_43f8_ab0d_3d2e961b2890.slice/crio-a1b6504a5edc6be677f88ddb1bfd25ed60090a4533b075915f4577242903858d WatchSource:0}: Error finding container a1b6504a5edc6be677f88ddb1bfd25ed60090a4533b075915f4577242903858d: Status 404 returned error can't find the container with id a1b6504a5edc6be677f88ddb1bfd25ed60090a4533b075915f4577242903858d
Jul 16 05:12:48.631244 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:48.631213    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs" event={"ID":"f91ccbe1-378f-43f8-ab0d-3d2e961b2890","Type":"ContainerStarted","Data":"5de2536dfcd28638c3b0c4464c65b42feddb6bc9e339483218643bd74e0d9c1f"}
Jul 16 05:12:48.631601 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:48.631261    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs" event={"ID":"f91ccbe1-378f-43f8-ab0d-3d2e961b2890","Type":"ContainerStarted","Data":"a1b6504a5edc6be677f88ddb1bfd25ed60090a4533b075915f4577242903858d"}
Jul 16 05:12:48.631601 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:48.631311    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:12:48.651448 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:48.651404    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs" podStartSLOduration=1.6513922220000001 podStartE2EDuration="1.651392222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:12:48.648856958 +0000 UTC m=+1365.569439752" watchObservedRunningTime="2026-07-16 05:12:48.651392222 +0000 UTC m=+1365.571975087"
Jul 16 05:12:59.638724 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:12:59.638694    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4dscs"
Jul 16 05:15:00.136486 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.136445    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"]
Jul 16 05:15:00.140136 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.140116    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"
Jul 16 05:15:00.142493 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.142466    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:15:00.150452 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.150432    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"
Jul 16 05:15:00.150691 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.150666    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"]
Jul 16 05:15:00.286537 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.286511    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"]
Jul 16 05:15:00.288815 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:15:00.288776    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62813987_835e_4b47_9af2_36b857ff6c64.slice/crio-dafaad4d3ca6735cdac0404fd3d3142a8ccc3311a32a8341bee31e11efc21272 WatchSource:0}: Error finding container dafaad4d3ca6735cdac0404fd3d3142a8ccc3311a32a8341bee31e11efc21272: Status 404 returned error can't find the container with id dafaad4d3ca6735cdac0404fd3d3142a8ccc3311a32a8341bee31e11efc21272
Jul 16 05:15:00.291030 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:00.291006    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 05:15:01.291558 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:01.291515    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2" event={"ID":"62813987-835e-4b47-9af2-36b857ff6c64","Type":"ContainerStarted","Data":"dafaad4d3ca6735cdac0404fd3d3142a8ccc3311a32a8341bee31e11efc21272"}
Jul 16 05:15:04.297645 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:04.297532    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:15:04.307367 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:04.298800    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:15:04.310617 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:04.310592    2565 generic.go:356] "Generic (PLEG): container finished" podID="62813987-835e-4b47-9af2-36b857ff6c64" containerID="9493e98254ff33af19dffec084edeb0c2346b3e9d313014cdc0e5365e70119da" exitCode=0
Jul 16 05:15:04.310725 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:04.310649    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2" event={"ID":"62813987-835e-4b47-9af2-36b857ff6c64","Type":"ContainerDied","Data":"9493e98254ff33af19dffec084edeb0c2346b3e9d313014cdc0e5365e70119da"}
Jul 16 05:15:05.455458 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:05.455430    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"
Jul 16 05:15:06.321011 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:06.320970    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2" event={"ID":"62813987-835e-4b47-9af2-36b857ff6c64","Type":"ContainerDied","Data":"dafaad4d3ca6735cdac0404fd3d3142a8ccc3311a32a8341bee31e11efc21272"}
Jul 16 05:15:06.321011 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:06.321006    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736315-wcnr2"
Jul 16 05:15:06.321215 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:15:06.321027    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dafaad4d3ca6735cdac0404fd3d3142a8ccc3311a32a8341bee31e11efc21272"
Jul 16 05:16:47.502679 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:16:47.502634    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:17:13.458915 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:17:13.457786    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:17:22.630922 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:17:22.630886    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:18:45.239298 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:18:45.239254    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:18:54.308653 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:18:54.308615    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:02.580213 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:02.580170    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:07.710621 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:07.710587    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:10.860100 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:10.860061    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:10.960077 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:10.960045    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:11.366179 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:11.366143    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:17.147617 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:17.147582    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:17.213162 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:17.213125    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:17.357587 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:17.357545    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:17.456103 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:17.455978    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:18.452923 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.452884    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:18.549196 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.549157    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"]
Jul 16 05:19:18.549636 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.549622    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62813987-835e-4b47-9af2-36b857ff6c64" containerName="cleanup"
Jul 16 05:19:18.549697 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.549638    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="62813987-835e-4b47-9af2-36b857ff6c64" containerName="cleanup"
Jul 16 05:19:18.549737 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.549709    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="62813987-835e-4b47-9af2-36b857ff6c64" containerName="cleanup"
Jul 16 05:19:18.550762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.550738    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:18.553244 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.553224    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.555655 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.555634    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f2ca692b-serving-cert\""
Jul 16 05:19:18.563341 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.563307    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"]
Jul 16 05:19:18.609566 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.609534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-openshift-service-ca\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.609772 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.609598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.609772 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.609636    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7scjn\" (UniqueName: \"kubernetes.io/projected/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-kube-api-access-7scjn\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.615201 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.615170    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:18.710767 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.710657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.710767 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.710699    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7scjn\" (UniqueName: \"kubernetes.io/projected/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-kube-api-access-7scjn\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.711062 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.710773    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-openshift-service-ca\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.711062 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:19:18.710867    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-serving-cert: secret "maas-api-e2e-ait-f2ca692b-serving-cert" not found
Jul 16 05:19:18.711062 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:19:18.711048    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls podName:f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b nodeName:}" failed. No retries permitted until 2026-07-16 05:19:19.211022479 +0000 UTC m=+1756.131605267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls") pod "maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" (UID: "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b") : secret "maas-api-e2e-ait-f2ca692b-serving-cert" not found
Jul 16 05:19:18.711498 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.711478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-openshift-service-ca\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:18.720070 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:18.720043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7scjn\" (UniqueName: \"kubernetes.io/projected/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-kube-api-access-7scjn\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:19.117687 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.117648    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:19.215844 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.215803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:19.217914 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.217821    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:19.218643 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.218619    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls\") pod \"maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:19.290500 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.290464    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:19.465177 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.465079    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:19.623649 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:19.623485    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"]
Jul 16 05:19:20.628569 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:20.628531    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" event={"ID":"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b","Type":"ContainerStarted","Data":"659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384"}
Jul 16 05:19:20.628984 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:20.628617    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" event={"ID":"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b","Type":"ContainerStarted","Data":"4cdc69b2a0403faf301da2c6bb5559bc7de84666c805a26a5e071a3b9a137c7a"}
Jul 16 05:19:20.628984 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:20.628679    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:20.646052 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:20.645998    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" podStartSLOduration=2.191418376 podStartE2EDuration="2.645983071s" totalImagesPullingTime="454.564695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:19:20.644579071 +0000 UTC m=+1757.565161868" watchObservedRunningTime="2026-07-16 05:19:20.645983071 +0000 UTC m=+1757.566565944"
Jul 16 05:19:21.360297 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:21.360260    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:19:23.619709 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.619667    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"]
Jul 16 05:19:23.623601 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.623574    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:23.626196 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.626171    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:19:23.628872 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.628809    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"]
Jul 16 05:19:23.655478 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.655447    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e3d753-741b-42a9-a221-38172e1345e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz\" (UID: \"79e3d753-741b-42a9-a221-38172e1345e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:23.756265 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.756214    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e3d753-741b-42a9-a221-38172e1345e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz\" (UID: \"79e3d753-741b-42a9-a221-38172e1345e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:23.756904 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.756879    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e3d753-741b-42a9-a221-38172e1345e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz\" (UID: \"79e3d753-741b-42a9-a221-38172e1345e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:23.937575 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:23.937490    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:24.089495 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:24.089468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"]
Jul 16 05:19:24.092534 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:19:24.092504    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e3d753_741b_42a9_a221_38172e1345e7.slice/crio-c8060d933c974fa951615bf0bad96990b6665e31de4e3ed20657da3e6366b89e WatchSource:0}: Error finding container c8060d933c974fa951615bf0bad96990b6665e31de4e3ed20657da3e6366b89e: Status 404 returned error can't find the container with id c8060d933c974fa951615bf0bad96990b6665e31de4e3ed20657da3e6366b89e
Jul 16 05:19:24.651071 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:24.651036    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" event={"ID":"79e3d753-741b-42a9-a221-38172e1345e7","Type":"ContainerStarted","Data":"accb8b6036c721ff55b32866a411b27f1f6b2c9f562dab6e6bf515bc74cf9548"}
Jul 16 05:19:24.651434 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:24.651090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" event={"ID":"79e3d753-741b-42a9-a221-38172e1345e7","Type":"ContainerStarted","Data":"c8060d933c974fa951615bf0bad96990b6665e31de4e3ed20657da3e6366b89e"}
Jul 16 05:19:24.666668 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:24.666615    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" podStartSLOduration=1.666597214 podStartE2EDuration="1.666597214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:19:24.664489407 +0000 UTC m=+1761.585072193" watchObservedRunningTime="2026-07-16 05:19:24.666597214 +0000 UTC m=+1761.587180009"
Jul 16 05:19:25.657405 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:25.657370    2565 generic.go:356] "Generic (PLEG): container finished" podID="79e3d753-741b-42a9-a221-38172e1345e7" containerID="accb8b6036c721ff55b32866a411b27f1f6b2c9f562dab6e6bf515bc74cf9548" exitCode=7
Jul 16 05:19:25.657807 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:25.657460    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" event={"ID":"79e3d753-741b-42a9-a221-38172e1345e7","Type":"ContainerDied","Data":"accb8b6036c721ff55b32866a411b27f1f6b2c9f562dab6e6bf515bc74cf9548"}
Jul 16 05:19:25.657807 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:25.657596    2565 scope.go:117] "RemoveContainer" containerID="accb8b6036c721ff55b32866a411b27f1f6b2c9f562dab6e6bf515bc74cf9548"
Jul 16 05:19:26.640391 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:26.640305    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:19:26.665710 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:26.665674    2565 generic.go:356] "Generic (PLEG): container finished" podID="79e3d753-741b-42a9-a221-38172e1345e7" containerID="ffa79c31dcf3425fac3e6117a7d9effa25dfcab6cf5461ccfd058e20d03e9ed1" exitCode=7
Jul 16 05:19:26.666186 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:26.665789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" event={"ID":"79e3d753-741b-42a9-a221-38172e1345e7","Type":"ContainerDied","Data":"ffa79c31dcf3425fac3e6117a7d9effa25dfcab6cf5461ccfd058e20d03e9ed1"}
Jul 16 05:19:26.666186 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:26.665891    2565 scope.go:117] "RemoveContainer" containerID="accb8b6036c721ff55b32866a411b27f1f6b2c9f562dab6e6bf515bc74cf9548"
Jul 16 05:19:26.666186 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:26.666098    2565 scope.go:117] "RemoveContainer" containerID="ffa79c31dcf3425fac3e6117a7d9effa25dfcab6cf5461ccfd058e20d03e9ed1"
Jul 16 05:19:26.666416 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:19:26.666389    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz_odh-ai-gateway-infra(79e3d753-741b-42a9-a221-38172e1345e7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" podUID="79e3d753-741b-42a9-a221-38172e1345e7"
Jul 16 05:19:27.673405 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:27.673321    2565 scope.go:117] "RemoveContainer" containerID="ffa79c31dcf3425fac3e6117a7d9effa25dfcab6cf5461ccfd058e20d03e9ed1"
Jul 16 05:19:27.673752 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:19:27.673537    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz_odh-ai-gateway-infra(79e3d753-741b-42a9-a221-38172e1345e7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" podUID="79e3d753-741b-42a9-a221-38172e1345e7"
Jul 16 05:19:42.645178 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:42.645140    2565 scope.go:117] "RemoveContainer" containerID="ffa79c31dcf3425fac3e6117a7d9effa25dfcab6cf5461ccfd058e20d03e9ed1"
Jul 16 05:19:43.756843 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:43.756800    2565 generic.go:356] "Generic (PLEG): container finished" podID="79e3d753-741b-42a9-a221-38172e1345e7" containerID="7f19a1024ffdeabd1e50b5957297b6e0ed536c54639a52c96e9ae913d0aeeb06" exitCode=0
Jul 16 05:19:43.757268 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:43.756871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" event={"ID":"79e3d753-741b-42a9-a221-38172e1345e7","Type":"ContainerDied","Data":"7f19a1024ffdeabd1e50b5957297b6e0ed536c54639a52c96e9ae913d0aeeb06"}
Jul 16 05:19:43.757268 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:43.756932    2565 scope.go:117] "RemoveContainer" containerID="ffa79c31dcf3425fac3e6117a7d9effa25dfcab6cf5461ccfd058e20d03e9ed1"
Jul 16 05:19:44.780594 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:44.780562    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"]
Jul 16 05:19:44.918147 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:44.918121    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:45.063570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.063484    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e3d753-741b-42a9-a221-38172e1345e7-maas-api-service-ca\") pod \"79e3d753-741b-42a9-a221-38172e1345e7\" (UID: \"79e3d753-741b-42a9-a221-38172e1345e7\") "
Jul 16 05:19:45.063849 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.063809    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e3d753-741b-42a9-a221-38172e1345e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "79e3d753-741b-42a9-a221-38172e1345e7" (UID: "79e3d753-741b-42a9-a221-38172e1345e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:19:45.165055 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.165015    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e3d753-741b-42a9-a221-38172e1345e7-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:19:45.778336 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.778312    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"
Jul 16 05:19:45.778508 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.778340    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz" event={"ID":"79e3d753-741b-42a9-a221-38172e1345e7","Type":"ContainerDied","Data":"c8060d933c974fa951615bf0bad96990b6665e31de4e3ed20657da3e6366b89e"}
Jul 16 05:19:45.778508 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.778384    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8060d933c974fa951615bf0bad96990b6665e31de4e3ed20657da3e6366b89e"
Jul 16 05:19:45.804215 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.804185    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"]
Jul 16 05:19:45.806279 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:45.806254    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-t77fz"]
Jul 16 05:19:47.651570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:19:47.651539    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e3d753-741b-42a9-a221-38172e1345e7" path="/var/lib/kubelet/pods/79e3d753-741b-42a9-a221-38172e1345e7/volumes"
Jul 16 05:20:04.329543 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:04.329513    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:20:04.332687 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:04.332337    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:20:23.663377 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.663341    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"]
Jul 16 05:20:23.663944 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.663929    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.663994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.663947    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.663994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.663962    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.663994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.663971    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.664086 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.664076    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.664118 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.664087    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.664118 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.664096    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:23.667676 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.667657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:23.670282 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.670254    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:20:23.673027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.672942    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"]
Jul 16 05:20:23.715958 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.715919    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b8e867-2d59-4d0d-80be-6be4731356b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m\" (UID: \"29b8e867-2d59-4d0d-80be-6be4731356b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:23.817008 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.816969    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b8e867-2d59-4d0d-80be-6be4731356b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m\" (UID: \"29b8e867-2d59-4d0d-80be-6be4731356b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:23.817597 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.817577    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b8e867-2d59-4d0d-80be-6be4731356b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m\" (UID: \"29b8e867-2d59-4d0d-80be-6be4731356b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:23.981965 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:23.981881    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:24.135076 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:24.135044    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"]
Jul 16 05:20:24.136918 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:20:24.136884    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b8e867_2d59_4d0d_80be_6be4731356b3.slice/crio-0536a744fcc1aa81f350d814c179ae41895992044e400e4afc78f2c8c6ce99be WatchSource:0}: Error finding container 0536a744fcc1aa81f350d814c179ae41895992044e400e4afc78f2c8c6ce99be: Status 404 returned error can't find the container with id 0536a744fcc1aa81f350d814c179ae41895992044e400e4afc78f2c8c6ce99be
Jul 16 05:20:24.977648 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:24.977615    2565 generic.go:356] "Generic (PLEG): container finished" podID="29b8e867-2d59-4d0d-80be-6be4731356b3" containerID="da4d47a0d90c3bf57a3531735afe1151e77af162868560ceb2c452a74000ab3a" exitCode=0
Jul 16 05:20:24.978069 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:24.977660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m" event={"ID":"29b8e867-2d59-4d0d-80be-6be4731356b3","Type":"ContainerDied","Data":"da4d47a0d90c3bf57a3531735afe1151e77af162868560ceb2c452a74000ab3a"}
Jul 16 05:20:24.978069 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:24.977723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m" event={"ID":"29b8e867-2d59-4d0d-80be-6be4731356b3","Type":"ContainerStarted","Data":"0536a744fcc1aa81f350d814c179ae41895992044e400e4afc78f2c8c6ce99be"}
Jul 16 05:20:26.129720 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.129695    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:26.237734 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.237634    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b8e867-2d59-4d0d-80be-6be4731356b3-maas-api-service-ca\") pod \"29b8e867-2d59-4d0d-80be-6be4731356b3\" (UID: \"29b8e867-2d59-4d0d-80be-6be4731356b3\") "
Jul 16 05:20:26.238112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.238069    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b8e867-2d59-4d0d-80be-6be4731356b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29b8e867-2d59-4d0d-80be-6be4731356b3" (UID: "29b8e867-2d59-4d0d-80be-6be4731356b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:20:26.339303 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.339264    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b8e867-2d59-4d0d-80be-6be4731356b3-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:20:26.988677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.988640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m" event={"ID":"29b8e867-2d59-4d0d-80be-6be4731356b3","Type":"ContainerDied","Data":"0536a744fcc1aa81f350d814c179ae41895992044e400e4afc78f2c8c6ce99be"}
Jul 16 05:20:26.988677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.988664    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m"
Jul 16 05:20:26.988939 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:26.988690    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0536a744fcc1aa81f350d814c179ae41895992044e400e4afc78f2c8c6ce99be"
Jul 16 05:20:34.015647 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.015603    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"]
Jul 16 05:20:34.016285 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.015905    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" podUID="f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" containerName="maas-api" containerID="cri-o://659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384" gracePeriod=30
Jul 16 05:20:34.274956 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.274892    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:20:34.421593 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.421553    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls\") pod \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") "
Jul 16 05:20:34.421802 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.421648    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7scjn\" (UniqueName: \"kubernetes.io/projected/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-kube-api-access-7scjn\") pod \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") "
Jul 16 05:20:34.421802 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.421672    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-openshift-service-ca\") pod \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\" (UID: \"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b\") "
Jul 16 05:20:34.422124 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.422097    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" (UID: "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:20:34.424128 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.424103    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-kube-api-access-7scjn" (OuterVolumeSpecName: "kube-api-access-7scjn") pod "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" (UID: "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b"). InnerVolumeSpecName "kube-api-access-7scjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:20:34.424201 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.424171    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" (UID: "f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:20:34.445400 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.445359    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:34.523252 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.523209    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:20:34.523252 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.523245    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7scjn\" (UniqueName: \"kubernetes.io/projected/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-kube-api-access-7scjn\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:20:34.523252 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:34.523256    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:20:35.034779 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.034746    2565 generic.go:356] "Generic (PLEG): container finished" podID="f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" containerID="659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384" exitCode=0
Jul 16 05:20:35.035275 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.034791    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"
Jul 16 05:20:35.035275 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.034858    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" event={"ID":"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b","Type":"ContainerDied","Data":"659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384"}
Jul 16 05:20:35.035275 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.034913    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6" event={"ID":"f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b","Type":"ContainerDied","Data":"4cdc69b2a0403faf301da2c6bb5559bc7de84666c805a26a5e071a3b9a137c7a"}
Jul 16 05:20:35.035275 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.034935    2565 scope.go:117] "RemoveContainer" containerID="659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384"
Jul 16 05:20:35.044259 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.044241    2565 scope.go:117] "RemoveContainer" containerID="659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384"
Jul 16 05:20:35.044578 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:20:35.044559    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384\": container with ID starting with 659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384 not found: ID does not exist" containerID="659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384"
Jul 16 05:20:35.044628 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.044591    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384"} err="failed to get container status \"659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384\": rpc error: code = NotFound desc = could not find container \"659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384\": container with ID starting with 659b52f589d7b43be1b5a086732c01cf0195887734ae588d5a67ddedd3860384 not found: ID does not exist"
Jul 16 05:20:35.055320 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.055288    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"]
Jul 16 05:20:35.059208 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.059181    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2ca692b-7c94b6c6d8-2sxd6"]
Jul 16 05:20:35.651398 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:35.651357    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" path="/var/lib/kubelet/pods/f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b/volumes"
Jul 16 05:20:38.734464 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:38.734424    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:44.557634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:44.557594    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:44.614225 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:44.614191    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:48.938990 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:48.938952    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:49.381620 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:49.381581    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:49.480683 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:49.480652    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:49.610572 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:49.610540    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:50.897636 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:50.897570    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:51.138544 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.137744    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"]
Jul 16 05:20:51.138544 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138471    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" containerName="maas-api"
Jul 16 05:20:51.138544 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138493    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" containerName="maas-api"
Jul 16 05:20:51.138544 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138528    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:51.138544 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138537    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e3d753-741b-42a9-a221-38172e1345e7" containerName="revoke-keys"
Jul 16 05:20:51.138544 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138550    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b8e867-2d59-4d0d-80be-6be4731356b3" containerName="revoke-keys"
Jul 16 05:20:51.138990 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138561    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b8e867-2d59-4d0d-80be-6be4731356b3" containerName="revoke-keys"
Jul 16 05:20:51.138990 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138660    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b67a98-08e1-4bf5-aeb4-6f7a1040ce0b" containerName="maas-api"
Jul 16 05:20:51.138990 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.138675    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b8e867-2d59-4d0d-80be-6be4731356b3" containerName="revoke-keys"
Jul 16 05:20:51.143471 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.143452    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.145536 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.145514    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-1eae9b50-serving-cert\""
Jul 16 05:20:51.150007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.149938    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"]
Jul 16 05:20:51.164580 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.164545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25vwj\" (UniqueName: \"kubernetes.io/projected/056880ce-f820-4670-9b06-ccee262d17d8-kube-api-access-25vwj\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.164719 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.164669    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.164719 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.164706    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/056880ce-f820-4670-9b06-ccee262d17d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.172560 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.172533    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:51.266014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.265985    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.266014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.266019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/056880ce-f820-4670-9b06-ccee262d17d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.266248 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.266092    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25vwj\" (UniqueName: \"kubernetes.io/projected/056880ce-f820-4670-9b06-ccee262d17d8-kube-api-access-25vwj\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.266248 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:20:51.266152    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-serving-cert: secret "maas-api-e2e-migrate-1eae9b50-serving-cert" not found
Jul 16 05:20:51.266248 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:20:51.266220    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls podName:056880ce-f820-4670-9b06-ccee262d17d8 nodeName:}" failed. No retries permitted until 2026-07-16 05:20:51.76620251 +0000 UTC m=+1848.686785285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls") pod "maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" (UID: "056880ce-f820-4670-9b06-ccee262d17d8") : secret "maas-api-e2e-migrate-1eae9b50-serving-cert" not found
Jul 16 05:20:51.266703 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.266685    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/056880ce-f820-4670-9b06-ccee262d17d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.273988 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.273962    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25vwj\" (UniqueName: \"kubernetes.io/projected/056880ce-f820-4670-9b06-ccee262d17d8-kube-api-access-25vwj\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.374293 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.374262    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:51.601124 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.601096    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:51.702480 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.702441    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:51.770501 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.770451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:51.773185 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:51.773158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls\") pod \"maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:52.057024 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:52.056933    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:52.212644 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:52.212612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"]
Jul 16 05:20:52.214792 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:20:52.214759    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056880ce_f820_4670_9b06_ccee262d17d8.slice/crio-0c19d86f7e3c81cd7cbc706c4f95f36a0522fcc18999ac34b21be9c0eede35bd WatchSource:0}: Error finding container 0c19d86f7e3c81cd7cbc706c4f95f36a0522fcc18999ac34b21be9c0eede35bd: Status 404 returned error can't find the container with id 0c19d86f7e3c81cd7cbc706c4f95f36a0522fcc18999ac34b21be9c0eede35bd
Jul 16 05:20:52.216107 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:52.216089    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 05:20:53.131462 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:53.131423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" event={"ID":"056880ce-f820-4670-9b06-ccee262d17d8","Type":"ContainerStarted","Data":"6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f"}
Jul 16 05:20:53.131885 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:53.131484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" event={"ID":"056880ce-f820-4670-9b06-ccee262d17d8","Type":"ContainerStarted","Data":"0c19d86f7e3c81cd7cbc706c4f95f36a0522fcc18999ac34b21be9c0eede35bd"}
Jul 16 05:20:53.131885 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:53.131510    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:53.148237 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:53.148183    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" podStartSLOduration=1.711279121 podStartE2EDuration="2.14816324s" totalImagesPullingTime="436.884119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:20:53.145667331 +0000 UTC m=+1850.066250139" watchObservedRunningTime="2026-07-16 05:20:53.14816324 +0000 UTC m=+1850.068746088"
Jul 16 05:20:55.368630 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.368592    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"]
Jul 16 05:20:55.372427 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.372403    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:20:55.374775 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.374749    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:20:55.377940 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.377913    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"]
Jul 16 05:20:55.403403 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.403369    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c3f60b-deba-4be3-b46f-75d1fd6406ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8\" (UID: \"b1c3f60b-deba-4be3-b46f-75d1fd6406ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:20:55.504534 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.504497    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c3f60b-deba-4be3-b46f-75d1fd6406ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8\" (UID: \"b1c3f60b-deba-4be3-b46f-75d1fd6406ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:20:55.505286 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.505265    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c3f60b-deba-4be3-b46f-75d1fd6406ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8\" (UID: \"b1c3f60b-deba-4be3-b46f-75d1fd6406ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:20:55.688346 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.688248    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:20:55.838632 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:55.838603    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"]
Jul 16 05:20:55.842048 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:20:55.842017    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c3f60b_deba_4be3_b46f_75d1fd6406ab.slice/crio-e92eccdcd0d7c64f4500974fa199b93aee20ed7a7992e5a47239189dd4d287f2 WatchSource:0}: Error finding container e92eccdcd0d7c64f4500974fa199b93aee20ed7a7992e5a47239189dd4d287f2: Status 404 returned error can't find the container with id e92eccdcd0d7c64f4500974fa199b93aee20ed7a7992e5a47239189dd4d287f2
Jul 16 05:20:56.149162 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:56.149124    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerStarted","Data":"858fa1839c1e9401179bcb4158c1a3e47b534dfba9228b910221cc9ce338cc17"}
Jul 16 05:20:56.149356 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:56.149179    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerStarted","Data":"e92eccdcd0d7c64f4500974fa199b93aee20ed7a7992e5a47239189dd4d287f2"}
Jul 16 05:20:56.163711 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:56.163657    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" podStartSLOduration=1.163639966 podStartE2EDuration="1.163639966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:20:56.162252185 +0000 UTC m=+1853.082834982" watchObservedRunningTime="2026-07-16 05:20:56.163639966 +0000 UTC m=+1853.084222807"
Jul 16 05:20:56.248956 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:56.248924    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:56.302926 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:56.302890    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:20:57.156358 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:57.156326    2565 generic.go:356] "Generic (PLEG): container finished" podID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerID="858fa1839c1e9401179bcb4158c1a3e47b534dfba9228b910221cc9ce338cc17" exitCode=7
Jul 16 05:20:57.156748 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:57.156423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerDied","Data":"858fa1839c1e9401179bcb4158c1a3e47b534dfba9228b910221cc9ce338cc17"}
Jul 16 05:20:57.156748 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:57.156573    2565 scope.go:117] "RemoveContainer" containerID="858fa1839c1e9401179bcb4158c1a3e47b534dfba9228b910221cc9ce338cc17"
Jul 16 05:20:58.163288 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:58.163192    2565 generic.go:356] "Generic (PLEG): container finished" podID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerID="a85e1cf4e3e37c3750ab802a2aed83c28dc5bfa1a9803ef7bc095274f19a2c17" exitCode=7
Jul 16 05:20:58.163677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:58.163286    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerDied","Data":"a85e1cf4e3e37c3750ab802a2aed83c28dc5bfa1a9803ef7bc095274f19a2c17"}
Jul 16 05:20:58.163677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:58.163348    2565 scope.go:117] "RemoveContainer" containerID="858fa1839c1e9401179bcb4158c1a3e47b534dfba9228b910221cc9ce338cc17"
Jul 16 05:20:58.163677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:58.163509    2565 scope.go:117] "RemoveContainer" containerID="a85e1cf4e3e37c3750ab802a2aed83c28dc5bfa1a9803ef7bc095274f19a2c17"
Jul 16 05:20:58.163889 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:20:58.163698    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8_odh-ai-gateway-infra(b1c3f60b-deba-4be3-b46f-75d1fd6406ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab"
Jul 16 05:20:59.142962 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:59.142920    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:20:59.172161 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:59.172126    2565 scope.go:117] "RemoveContainer" containerID="a85e1cf4e3e37c3750ab802a2aed83c28dc5bfa1a9803ef7bc095274f19a2c17"
Jul 16 05:20:59.172570 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:20:59.172400    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8_odh-ai-gateway-infra(b1c3f60b-deba-4be3-b46f-75d1fd6406ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab"
Jul 16 05:20:59.699761 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:20:59.699724    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:21:11.645090 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:11.645050    2565 scope.go:117] "RemoveContainer" containerID="a85e1cf4e3e37c3750ab802a2aed83c28dc5bfa1a9803ef7bc095274f19a2c17"
Jul 16 05:21:12.241024 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:12.240991    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerStarted","Data":"913ba4a65a99d2ded4ac95cbe843871ff16c53f835911dd65c075e0501e539f3"}
Jul 16 05:21:13.248691 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:13.248657    2565 generic.go:356] "Generic (PLEG): container finished" podID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerID="913ba4a65a99d2ded4ac95cbe843871ff16c53f835911dd65c075e0501e539f3" exitCode=0
Jul 16 05:21:13.249138 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:13.248727    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerDied","Data":"913ba4a65a99d2ded4ac95cbe843871ff16c53f835911dd65c075e0501e539f3"}
Jul 16 05:21:13.249138 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:13.248788    2565 scope.go:117] "RemoveContainer" containerID="a85e1cf4e3e37c3750ab802a2aed83c28dc5bfa1a9803ef7bc095274f19a2c17"
Jul 16 05:21:13.263215 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:13.263189    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"]
Jul 16 05:21:14.398888 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:14.398861    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:21:14.490987 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:14.490950    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c3f60b-deba-4be3-b46f-75d1fd6406ab-maas-api-service-ca\") pod \"b1c3f60b-deba-4be3-b46f-75d1fd6406ab\" (UID: \"b1c3f60b-deba-4be3-b46f-75d1fd6406ab\") "
Jul 16 05:21:14.491350 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:14.491321    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1c3f60b-deba-4be3-b46f-75d1fd6406ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1c3f60b-deba-4be3-b46f-75d1fd6406ab" (UID: "b1c3f60b-deba-4be3-b46f-75d1fd6406ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:21:14.591972 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:14.591935    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c3f60b-deba-4be3-b46f-75d1fd6406ab-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:21:15.262707 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:15.262678    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"
Jul 16 05:21:15.263017 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:15.262672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8" event={"ID":"b1c3f60b-deba-4be3-b46f-75d1fd6406ab","Type":"ContainerDied","Data":"e92eccdcd0d7c64f4500974fa199b93aee20ed7a7992e5a47239189dd4d287f2"}
Jul 16 05:21:15.263017 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:15.262815    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e92eccdcd0d7c64f4500974fa199b93aee20ed7a7992e5a47239189dd4d287f2"
Jul 16 05:21:15.290760 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:15.290732    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"]
Jul 16 05:21:15.292677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:15.292658    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-lzjh8"]
Jul 16 05:21:15.651064 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:15.651030    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" path="/var/lib/kubelet/pods/b1c3f60b-deba-4be3-b46f-75d1fd6406ab/volumes"
Jul 16 05:21:31.665570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:31.665534    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:21:45.404397 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404357    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"]
Jul 16 05:21:45.404892 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404877    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:21:45.404952 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404896    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:21:45.404952 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404907    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:21:45.404952 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404914    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:21:45.405046 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404985    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:21:45.405046 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.404995    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:21:45.408226 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.408210    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:45.410731 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.410706    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:21:45.415027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.414995    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"]
Jul 16 05:21:45.489406 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.489366    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6303a1-6b39-4735-9475-ea30a32446b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z\" (UID: \"2e6303a1-6b39-4735-9475-ea30a32446b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:45.590261 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.590226    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6303a1-6b39-4735-9475-ea30a32446b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z\" (UID: \"2e6303a1-6b39-4735-9475-ea30a32446b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:45.590956 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.590932    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6303a1-6b39-4735-9475-ea30a32446b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z\" (UID: \"2e6303a1-6b39-4735-9475-ea30a32446b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:45.720943 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.720810    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:45.871404 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:45.871373    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"]
Jul 16 05:21:45.873804 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:21:45.873774    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6303a1_6b39_4735_9475_ea30a32446b1.slice/crio-3ed54248e4d2f9bb14c89b88be8a5870a21e60e0a7710c22753a06857b8fe9de WatchSource:0}: Error finding container 3ed54248e4d2f9bb14c89b88be8a5870a21e60e0a7710c22753a06857b8fe9de: Status 404 returned error can't find the container with id 3ed54248e4d2f9bb14c89b88be8a5870a21e60e0a7710c22753a06857b8fe9de
Jul 16 05:21:46.435066 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:46.434968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z" event={"ID":"2e6303a1-6b39-4735-9475-ea30a32446b1","Type":"ContainerStarted","Data":"f39c02a88083a854a2b772a843d52dbdccf79bd52799f061d552df3b1829aebb"}
Jul 16 05:21:46.435066 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:46.435028    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z" event={"ID":"2e6303a1-6b39-4735-9475-ea30a32446b1","Type":"ContainerStarted","Data":"3ed54248e4d2f9bb14c89b88be8a5870a21e60e0a7710c22753a06857b8fe9de"}
Jul 16 05:21:46.449101 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:46.449048    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z" podStartSLOduration=1.4490333930000001 podStartE2EDuration="1.449033393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:21:46.448466021 +0000 UTC m=+1903.369048819" watchObservedRunningTime="2026-07-16 05:21:46.449033393 +0000 UTC m=+1903.369616187"
Jul 16 05:21:47.442291 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:47.442257    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e6303a1-6b39-4735-9475-ea30a32446b1" containerID="f39c02a88083a854a2b772a843d52dbdccf79bd52799f061d552df3b1829aebb" exitCode=0
Jul 16 05:21:47.442701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:47.442345    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z" event={"ID":"2e6303a1-6b39-4735-9475-ea30a32446b1","Type":"ContainerDied","Data":"f39c02a88083a854a2b772a843d52dbdccf79bd52799f061d552df3b1829aebb"}
Jul 16 05:21:48.601112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:48.601082    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:48.719727 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:48.719631    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6303a1-6b39-4735-9475-ea30a32446b1-maas-api-service-ca\") pod \"2e6303a1-6b39-4735-9475-ea30a32446b1\" (UID: \"2e6303a1-6b39-4735-9475-ea30a32446b1\") "
Jul 16 05:21:48.720044 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:48.720021    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e6303a1-6b39-4735-9475-ea30a32446b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e6303a1-6b39-4735-9475-ea30a32446b1" (UID: "2e6303a1-6b39-4735-9475-ea30a32446b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:21:48.820615 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:48.820581    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6303a1-6b39-4735-9475-ea30a32446b1-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:21:49.453889 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:49.453860    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z"
Jul 16 05:21:49.454099 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:49.453859    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z" event={"ID":"2e6303a1-6b39-4735-9475-ea30a32446b1","Type":"ContainerDied","Data":"3ed54248e4d2f9bb14c89b88be8a5870a21e60e0a7710c22753a06857b8fe9de"}
Jul 16 05:21:49.454099 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:49.453998    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ed54248e4d2f9bb14c89b88be8a5870a21e60e0a7710c22753a06857b8fe9de"
Jul 16 05:21:55.535101 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:55.535052    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:21:55.830216 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:55.830182    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"]
Jul 16 05:21:55.830490 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:55.830435    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" podUID="056880ce-f820-4670-9b06-ccee262d17d8" containerName="maas-api" containerID="cri-o://6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f" gracePeriod=30
Jul 16 05:21:56.095431 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.095356    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:21:56.193990 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.193952    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25vwj\" (UniqueName: \"kubernetes.io/projected/056880ce-f820-4670-9b06-ccee262d17d8-kube-api-access-25vwj\") pod \"056880ce-f820-4670-9b06-ccee262d17d8\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") "
Jul 16 05:21:56.194182 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.194047    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls\") pod \"056880ce-f820-4670-9b06-ccee262d17d8\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") "
Jul 16 05:21:56.194182 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.194117    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/056880ce-f820-4670-9b06-ccee262d17d8-openshift-service-ca\") pod \"056880ce-f820-4670-9b06-ccee262d17d8\" (UID: \"056880ce-f820-4670-9b06-ccee262d17d8\") "
Jul 16 05:21:56.194511 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.194482    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/056880ce-f820-4670-9b06-ccee262d17d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "056880ce-f820-4670-9b06-ccee262d17d8" (UID: "056880ce-f820-4670-9b06-ccee262d17d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:21:56.196410 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.196384    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "056880ce-f820-4670-9b06-ccee262d17d8" (UID: "056880ce-f820-4670-9b06-ccee262d17d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:21:56.196483 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.196463    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056880ce-f820-4670-9b06-ccee262d17d8-kube-api-access-25vwj" (OuterVolumeSpecName: "kube-api-access-25vwj") pod "056880ce-f820-4670-9b06-ccee262d17d8" (UID: "056880ce-f820-4670-9b06-ccee262d17d8"). InnerVolumeSpecName "kube-api-access-25vwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:21:56.295364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.295321    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/056880ce-f820-4670-9b06-ccee262d17d8-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:21:56.295364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.295358    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/056880ce-f820-4670-9b06-ccee262d17d8-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:21:56.295364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.295368    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25vwj\" (UniqueName: \"kubernetes.io/projected/056880ce-f820-4670-9b06-ccee262d17d8-kube-api-access-25vwj\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:21:56.490933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.490815    2565 generic.go:356] "Generic (PLEG): container finished" podID="056880ce-f820-4670-9b06-ccee262d17d8" containerID="6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f" exitCode=0
Jul 16 05:21:56.490933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.490882    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"
Jul 16 05:21:56.490933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.490903    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" event={"ID":"056880ce-f820-4670-9b06-ccee262d17d8","Type":"ContainerDied","Data":"6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f"}
Jul 16 05:21:56.491165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.490962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698" event={"ID":"056880ce-f820-4670-9b06-ccee262d17d8","Type":"ContainerDied","Data":"0c19d86f7e3c81cd7cbc706c4f95f36a0522fcc18999ac34b21be9c0eede35bd"}
Jul 16 05:21:56.491165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.490986    2565 scope.go:117] "RemoveContainer" containerID="6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f"
Jul 16 05:21:56.501111 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.501092    2565 scope.go:117] "RemoveContainer" containerID="6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f"
Jul 16 05:21:56.501401 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:21:56.501381    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f\": container with ID starting with 6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f not found: ID does not exist" containerID="6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f"
Jul 16 05:21:56.501443 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.501412    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f"} err="failed to get container status \"6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f\": rpc error: code = NotFound desc = could not find container \"6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f\": container with ID starting with 6f4f5e035f4a5c2021657bc08ad32234f1e865b8c9e48ee6c05b7498e0dea65f not found: ID does not exist"
Jul 16 05:21:56.512403 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.512372    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"]
Jul 16 05:21:56.515928 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:56.515900    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1eae9b50-6759c9589d-vf698"]
Jul 16 05:21:57.158548 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:57.158515    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:21:57.651456 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:21:57.651422    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056880ce-f820-4670-9b06-ccee262d17d8" path="/var/lib/kubelet/pods/056880ce-f820-4670-9b06-ccee262d17d8/volumes"
Jul 16 05:22:01.482313 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:01.482275    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:01.582543 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:01.582509    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:01.724268 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:01.724237    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:01.989043 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:01.989007    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:02.092811 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.092773    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:02.845890 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.845846    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:02.921151 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921112    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"]
Jul 16 05:22:02.921625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921607    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056880ce-f820-4670-9b06-ccee262d17d8" containerName="maas-api"
Jul 16 05:22:02.921729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921628    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="056880ce-f820-4670-9b06-ccee262d17d8" containerName="maas-api"
Jul 16 05:22:02.921729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921652    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6303a1-6b39-4735-9475-ea30a32446b1" containerName="revoke-keys"
Jul 16 05:22:02.921729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921660    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6303a1-6b39-4735-9475-ea30a32446b1" containerName="revoke-keys"
Jul 16 05:22:02.921729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921675    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:22:02.921729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921684    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:22:02.921988 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921785    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e6303a1-6b39-4735-9475-ea30a32446b1" containerName="revoke-keys"
Jul 16 05:22:02.921988 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921798    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="056880ce-f820-4670-9b06-ccee262d17d8" containerName="maas-api"
Jul 16 05:22:02.921988 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.921810    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1c3f60b-deba-4be3-b46f-75d1fd6406ab" containerName="revoke-keys"
Jul 16 05:22:02.926309 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.926286    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:02.929317 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.929282    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-71d327bc-serving-cert\""
Jul 16 05:22:02.935420 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.935392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"]
Jul 16 05:22:02.936493 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.936467    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:03.000041 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:02.999995    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:03.061387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.061346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.061553 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.061410    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-openshift-service-ca\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.061553 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.061451    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2td\" (UniqueName: \"kubernetes.io/projected/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-kube-api-access-wx2td\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.162873 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.162749    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.163046 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.162905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-openshift-service-ca\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.163046 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:22:03.162930    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-serving-cert: secret "maas-api-e2e-ait-71d327bc-serving-cert" not found
Jul 16 05:22:03.163046 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.162986    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2td\" (UniqueName: \"kubernetes.io/projected/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-kube-api-access-wx2td\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.163046 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:22:03.163000    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls podName:b9f36e12-82bd-4fde-bc8b-4b7566d11ef2 nodeName:}" failed. No retries permitted until 2026-07-16 05:22:03.662982564 +0000 UTC m=+1920.583565337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls") pod "maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" (UID: "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2") : secret "maas-api-e2e-ait-71d327bc-serving-cert" not found
Jul 16 05:22:03.163649 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.163625    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-openshift-service-ca\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.173356 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.173327    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2td\" (UniqueName: \"kubernetes.io/projected/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-kube-api-access-wx2td\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.473526 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.473436    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:03.573941 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.573899    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:03.666677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.666648    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.669091 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.669073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls\") pod \"maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.674082 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.674052    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:03.839786 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.839752    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:03.991901 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:03.991873    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"]
Jul 16 05:22:03.994447 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:22:03.994417    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f36e12_82bd_4fde_bc8b_4b7566d11ef2.slice/crio-25cf5eb8b3b5934e1b2d63b2b8b7d65f8a9c22407711e5fe34d9aa4654151c92 WatchSource:0}: Error finding container 25cf5eb8b3b5934e1b2d63b2b8b7d65f8a9c22407711e5fe34d9aa4654151c92: Status 404 returned error can't find the container with id 25cf5eb8b3b5934e1b2d63b2b8b7d65f8a9c22407711e5fe34d9aa4654151c92
Jul 16 05:22:04.536001 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:04.535965    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" event={"ID":"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2","Type":"ContainerStarted","Data":"5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17"}
Jul 16 05:22:04.536208 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:04.536035    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:04.536208 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:04.536054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" event={"ID":"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2","Type":"ContainerStarted","Data":"25cf5eb8b3b5934e1b2d63b2b8b7d65f8a9c22407711e5fe34d9aa4654151c92"}
Jul 16 05:22:04.556094 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:04.556041    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" podStartSLOduration=2.11869118 podStartE2EDuration="2.556023609s" totalImagesPullingTime="437.332429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:22:04.553627164 +0000 UTC m=+1921.474209960" watchObservedRunningTime="2026-07-16 05:22:04.556023609 +0000 UTC m=+1921.476606405"
Jul 16 05:22:05.934494 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:05.934459    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:08.041369 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.041331    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"]
Jul 16 05:22:08.044883 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.044865    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:08.047126 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.047104    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:22:08.051562 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.051528    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"]
Jul 16 05:22:08.216962 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.216922    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78fdb13-4e0a-40f9-8556-8278b4d88fb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq\" (UID: \"b78fdb13-4e0a-40f9-8556-8278b4d88fb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:08.318330 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.318294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78fdb13-4e0a-40f9-8556-8278b4d88fb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq\" (UID: \"b78fdb13-4e0a-40f9-8556-8278b4d88fb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:08.318973 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.318953    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78fdb13-4e0a-40f9-8556-8278b4d88fb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq\" (UID: \"b78fdb13-4e0a-40f9-8556-8278b4d88fb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:08.356818 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.356783    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:08.505852 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.505808    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"]
Jul 16 05:22:08.507339 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:22:08.507312    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78fdb13_4e0a_40f9_8556_8278b4d88fb7.slice/crio-0ac525bb14ec484e3eafc50e78936b9ce53997c5c3f206e6f1964143408b29a0 WatchSource:0}: Error finding container 0ac525bb14ec484e3eafc50e78936b9ce53997c5c3f206e6f1964143408b29a0: Status 404 returned error can't find the container with id 0ac525bb14ec484e3eafc50e78936b9ce53997c5c3f206e6f1964143408b29a0
Jul 16 05:22:08.559922 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:08.559885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" event={"ID":"b78fdb13-4e0a-40f9-8556-8278b4d88fb7","Type":"ContainerStarted","Data":"0ac525bb14ec484e3eafc50e78936b9ce53997c5c3f206e6f1964143408b29a0"}
Jul 16 05:22:09.566583 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:09.566491    2565 generic.go:356] "Generic (PLEG): container finished" podID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerID="4069ed23bcdb5c34860f28951696bb748c79ff5ea6f5d22f653518697d6d962e" exitCode=7
Jul 16 05:22:09.566583 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:09.566536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" event={"ID":"b78fdb13-4e0a-40f9-8556-8278b4d88fb7","Type":"ContainerDied","Data":"4069ed23bcdb5c34860f28951696bb748c79ff5ea6f5d22f653518697d6d962e"}
Jul 16 05:22:09.567007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:09.566771    2565 scope.go:117] "RemoveContainer" containerID="4069ed23bcdb5c34860f28951696bb748c79ff5ea6f5d22f653518697d6d962e"
Jul 16 05:22:10.547732 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:10.547656    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:22:10.575230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:10.575196    2565 generic.go:356] "Generic (PLEG): container finished" podID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerID="fd8e2bc5b552e2519e5b29b5afc2820cc117e8314374a6eb1bd6cfdbb59b2e8f" exitCode=7
Jul 16 05:22:10.575714 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:10.575295    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" event={"ID":"b78fdb13-4e0a-40f9-8556-8278b4d88fb7","Type":"ContainerDied","Data":"fd8e2bc5b552e2519e5b29b5afc2820cc117e8314374a6eb1bd6cfdbb59b2e8f"}
Jul 16 05:22:10.575714 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:10.575381    2565 scope.go:117] "RemoveContainer" containerID="4069ed23bcdb5c34860f28951696bb748c79ff5ea6f5d22f653518697d6d962e"
Jul 16 05:22:10.575714 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:10.575708    2565 scope.go:117] "RemoveContainer" containerID="fd8e2bc5b552e2519e5b29b5afc2820cc117e8314374a6eb1bd6cfdbb59b2e8f"
Jul 16 05:22:10.576012 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:22:10.575987    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq_odh-ai-gateway-infra(b78fdb13-4e0a-40f9-8556-8278b4d88fb7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7"
Jul 16 05:22:11.582805 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:11.582779    2565 scope.go:117] "RemoveContainer" containerID="fd8e2bc5b552e2519e5b29b5afc2820cc117e8314374a6eb1bd6cfdbb59b2e8f"
Jul 16 05:22:11.583208 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:22:11.583025    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq_odh-ai-gateway-infra(b78fdb13-4e0a-40f9-8556-8278b4d88fb7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7"
Jul 16 05:22:15.736233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:15.736198    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:15.837473 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:15.837433    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:23.648294 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:23.648247    2565 scope.go:117] "RemoveContainer" containerID="fd8e2bc5b552e2519e5b29b5afc2820cc117e8314374a6eb1bd6cfdbb59b2e8f"
Jul 16 05:22:24.466762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:24.466730    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:24.567998 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:24.567964    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:22:24.656210 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:24.656175    2565 generic.go:356] "Generic (PLEG): container finished" podID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerID="74ab8cde932eefe90cb5910ee88eb8cae1b657f89b203003bdaa5dc66dfb3b04" exitCode=0
Jul 16 05:22:24.656642 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:24.656245    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" event={"ID":"b78fdb13-4e0a-40f9-8556-8278b4d88fb7","Type":"ContainerDied","Data":"74ab8cde932eefe90cb5910ee88eb8cae1b657f89b203003bdaa5dc66dfb3b04"}
Jul 16 05:22:24.656642 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:24.656317    2565 scope.go:117] "RemoveContainer" containerID="fd8e2bc5b552e2519e5b29b5afc2820cc117e8314374a6eb1bd6cfdbb59b2e8f"
Jul 16 05:22:25.686636 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:25.686605    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"]
Jul 16 05:22:25.803600 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:25.803574    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:25.880894 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:25.880853    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78fdb13-4e0a-40f9-8556-8278b4d88fb7-maas-api-service-ca\") pod \"b78fdb13-4e0a-40f9-8556-8278b4d88fb7\" (UID: \"b78fdb13-4e0a-40f9-8556-8278b4d88fb7\") "
Jul 16 05:22:25.881219 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:25.881196    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78fdb13-4e0a-40f9-8556-8278b4d88fb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b78fdb13-4e0a-40f9-8556-8278b4d88fb7" (UID: "b78fdb13-4e0a-40f9-8556-8278b4d88fb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:22:25.982621 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:25.982535    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b78fdb13-4e0a-40f9-8556-8278b4d88fb7-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:22:26.669666 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:26.669630    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"
Jul 16 05:22:26.669909 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:26.669673    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq" event={"ID":"b78fdb13-4e0a-40f9-8556-8278b4d88fb7","Type":"ContainerDied","Data":"0ac525bb14ec484e3eafc50e78936b9ce53997c5c3f206e6f1964143408b29a0"}
Jul 16 05:22:26.669909 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:26.669723    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ac525bb14ec484e3eafc50e78936b9ce53997c5c3f206e6f1964143408b29a0"
Jul 16 05:22:26.698787 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:26.698753    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"]
Jul 16 05:22:26.701108 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:26.701085    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-q8qmq"]
Jul 16 05:22:27.652359 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:27.652276    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" path="/var/lib/kubelet/pods/b78fdb13-4e0a-40f9-8556-8278b4d88fb7/volumes"
Jul 16 05:22:58.081272 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081184    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"]
Jul 16 05:22:58.081839 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081806    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.081893 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081846    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.081893 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081872    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.081893 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081880    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.081989 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081907    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.081989 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081915    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.082049 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.081998    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.082049 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.082008    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:22:58.085993 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.085972    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:22:58.088554 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.088530    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:22:58.091121 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.091096    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"]
Jul 16 05:22:58.188041 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.187998    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18a4f5-af2e-4a07-ab85-52d44eec174b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-71d327bc-xptbz\" (UID: \"ad18a4f5-af2e-4a07-ab85-52d44eec174b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:22:58.289077 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.289039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18a4f5-af2e-4a07-ab85-52d44eec174b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-71d327bc-xptbz\" (UID: \"ad18a4f5-af2e-4a07-ab85-52d44eec174b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:22:58.289711 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.289689    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18a4f5-af2e-4a07-ab85-52d44eec174b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-71d327bc-xptbz\" (UID: \"ad18a4f5-af2e-4a07-ab85-52d44eec174b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:22:58.400277 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.400180    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:22:58.544604 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.544571    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"]
Jul 16 05:22:58.546155 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:22:58.546118    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad18a4f5_af2e_4a07_ab85_52d44eec174b.slice/crio-21f40ba9bed75483de079fd73ee5753567830c4b46b2fa6f845087987b83ef6c WatchSource:0}: Error finding container 21f40ba9bed75483de079fd73ee5753567830c4b46b2fa6f845087987b83ef6c: Status 404 returned error can't find the container with id 21f40ba9bed75483de079fd73ee5753567830c4b46b2fa6f845087987b83ef6c
Jul 16 05:22:58.837193 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:58.837156    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz" event={"ID":"ad18a4f5-af2e-4a07-ab85-52d44eec174b","Type":"ContainerStarted","Data":"21f40ba9bed75483de079fd73ee5753567830c4b46b2fa6f845087987b83ef6c"}
Jul 16 05:22:59.844063 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:59.844029    2565 generic.go:356] "Generic (PLEG): container finished" podID="ad18a4f5-af2e-4a07-ab85-52d44eec174b" containerID="9c543bce45ca010142ce59f6cc0f18cc633ecd9aa7ba57089930508ca0fd51a6" exitCode=0
Jul 16 05:22:59.844459 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:22:59.844117    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz" event={"ID":"ad18a4f5-af2e-4a07-ab85-52d44eec174b","Type":"ContainerDied","Data":"9c543bce45ca010142ce59f6cc0f18cc633ecd9aa7ba57089930508ca0fd51a6"}
Jul 16 05:23:00.992246 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:00.992218    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:23:01.016201 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:01.016172    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18a4f5-af2e-4a07-ab85-52d44eec174b-maas-api-service-ca\") pod \"ad18a4f5-af2e-4a07-ab85-52d44eec174b\" (UID: \"ad18a4f5-af2e-4a07-ab85-52d44eec174b\") "
Jul 16 05:23:01.016624 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:01.016597    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad18a4f5-af2e-4a07-ab85-52d44eec174b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad18a4f5-af2e-4a07-ab85-52d44eec174b" (UID: "ad18a4f5-af2e-4a07-ab85-52d44eec174b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:23:01.117421 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:01.117325    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad18a4f5-af2e-4a07-ab85-52d44eec174b-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:23:01.855871 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:01.855842    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz"
Jul 16 05:23:01.855871 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:01.855853    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-71d327bc-xptbz" event={"ID":"ad18a4f5-af2e-4a07-ab85-52d44eec174b","Type":"ContainerDied","Data":"21f40ba9bed75483de079fd73ee5753567830c4b46b2fa6f845087987b83ef6c"}
Jul 16 05:23:01.856087 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:01.855903    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21f40ba9bed75483de079fd73ee5753567830c4b46b2fa6f845087987b83ef6c"
Jul 16 05:23:08.450494 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.450446    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"]
Jul 16 05:23:08.451020 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.450684    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" podUID="b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" containerName="maas-api" containerID="cri-o://5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17" gracePeriod=30
Jul 16 05:23:08.475866 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.475812    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:08.710412 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.710347    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:23:08.784160 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.784126    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-openshift-service-ca\") pod \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") "
Jul 16 05:23:08.784387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.784187    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls\") pod \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") "
Jul 16 05:23:08.784387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.784231    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx2td\" (UniqueName: \"kubernetes.io/projected/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-kube-api-access-wx2td\") pod \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\" (UID: \"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2\") "
Jul 16 05:23:08.784572 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.784549    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" (UID: "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:23:08.786492 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.786462    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" (UID: "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:23:08.786615 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.786531    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-kube-api-access-wx2td" (OuterVolumeSpecName: "kube-api-access-wx2td") pod "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" (UID: "b9f36e12-82bd-4fde-bc8b-4b7566d11ef2"). InnerVolumeSpecName "kube-api-access-wx2td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:23:08.885971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.885922    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:23:08.885971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.885957    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:23:08.885971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.885966    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx2td\" (UniqueName: \"kubernetes.io/projected/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2-kube-api-access-wx2td\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:23:08.893139 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.893103    2565 generic.go:356] "Generic (PLEG): container finished" podID="b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" containerID="5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17" exitCode=0
Jul 16 05:23:08.893324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.893157    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"
Jul 16 05:23:08.893324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.893189    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" event={"ID":"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2","Type":"ContainerDied","Data":"5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17"}
Jul 16 05:23:08.893324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.893280    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt" event={"ID":"b9f36e12-82bd-4fde-bc8b-4b7566d11ef2","Type":"ContainerDied","Data":"25cf5eb8b3b5934e1b2d63b2b8b7d65f8a9c22407711e5fe34d9aa4654151c92"}
Jul 16 05:23:08.893324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.893306    2565 scope.go:117] "RemoveContainer" containerID="5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17"
Jul 16 05:23:08.903820 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.903791    2565 scope.go:117] "RemoveContainer" containerID="5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17"
Jul 16 05:23:08.904259 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:23:08.904230    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17\": container with ID starting with 5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17 not found: ID does not exist" containerID="5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17"
Jul 16 05:23:08.904367 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.904281    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17"} err="failed to get container status \"5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17\": rpc error: code = NotFound desc = could not find container \"5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17\": container with ID starting with 5cbdcb44eca598612b019e4f26f414499e8fc46bef50ea6e521319445da85a17 not found: ID does not exist"
Jul 16 05:23:08.914217 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.914188    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"]
Jul 16 05:23:08.919498 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:08.919473    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-71d327bc-7b65f6bccb-c7pvt"]
Jul 16 05:23:09.652023 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:09.651993    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" path="/var/lib/kubelet/pods/b9f36e12-82bd-4fde-bc8b-4b7566d11ef2/volumes"
Jul 16 05:23:13.157373 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:13.157290    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:18.875679 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:18.875642    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:18.937368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:18.937333    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:24.383528 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:24.383491    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:24.760062 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:24.760029    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:25.884943 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:25.884901    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:25.983117 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:25.983079    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:26.126223 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.126180    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"]
Jul 16 05:23:26.126857 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.126817    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" containerName="maas-api"
Jul 16 05:23:26.126903 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.126862    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" containerName="maas-api"
Jul 16 05:23:26.126950 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.126908    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad18a4f5-af2e-4a07-ab85-52d44eec174b" containerName="revoke-keys"
Jul 16 05:23:26.126950 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.126917    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad18a4f5-af2e-4a07-ab85-52d44eec174b" containerName="revoke-keys"
Jul 16 05:23:26.127038 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.127025    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78fdb13-4e0a-40f9-8556-8278b4d88fb7" containerName="revoke-keys"
Jul 16 05:23:26.127072 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.127044    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad18a4f5-af2e-4a07-ab85-52d44eec174b" containerName="revoke-keys"
Jul 16 05:23:26.127072 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.127055    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9f36e12-82bd-4fde-bc8b-4b7566d11ef2" containerName="maas-api"
Jul 16 05:23:26.131546 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.131526    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.133770 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.133744    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-8142960b-serving-cert\""
Jul 16 05:23:26.140687 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.140664    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"]
Jul 16 05:23:26.159047 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.159015    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:26.249622 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.249581    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:26.259906 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.259875    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5xmn\" (UniqueName: \"kubernetes.io/projected/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-kube-api-access-m5xmn\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.260061 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.259931    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-maas-api-tls\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.260061 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.259979    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-openshift-service-ca\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.361264 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.361230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5xmn\" (UniqueName: \"kubernetes.io/projected/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-kube-api-access-m5xmn\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.361445 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.361292    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-maas-api-tls\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.361445 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.361325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-openshift-service-ca\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.362060 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.362034    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-openshift-service-ca\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.371552 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.371521    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-maas-api-tls\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.371817 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.371798    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5xmn\" (UniqueName: \"kubernetes.io/projected/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-kube-api-access-m5xmn\") pod \"maas-api-e2e-derive-8142960b-54b6bc64f8-j9292\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.444687 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.444593    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:26.593925 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.593899    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"]
Jul 16 05:23:26.596943 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:23:26.596885    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c3bf6d_f8d1_4dbc_b292_b4f3ad4a0bbe.slice/crio-5a2627ffd697c09b913ffbcf86f1edd2e76cadb747cc11672201b252e6d89a0e WatchSource:0}: Error finding container 5a2627ffd697c09b913ffbcf86f1edd2e76cadb747cc11672201b252e6d89a0e: Status 404 returned error can't find the container with id 5a2627ffd697c09b913ffbcf86f1edd2e76cadb747cc11672201b252e6d89a0e
Jul 16 05:23:26.642647 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.642611    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:26.745784 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.745706    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:26.992719 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:26.992674    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292" event={"ID":"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe","Type":"ContainerStarted","Data":"5a2627ffd697c09b913ffbcf86f1edd2e76cadb747cc11672201b252e6d89a0e"}
Jul 16 05:23:27.999335 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:27.999305    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292" event={"ID":"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe","Type":"ContainerStarted","Data":"f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade"}
Jul 16 05:23:27.999736 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:27.999358    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:28.017814 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:28.017758    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292" podStartSLOduration=1.667920948 podStartE2EDuration="2.017745067s" totalImagesPullingTime="349.824119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:23:28.016212948 +0000 UTC m=+2004.936795743" watchObservedRunningTime="2026-07-16 05:23:28.017745067 +0000 UTC m=+2004.938327862"
Jul 16 05:23:30.393320 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.393277    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"]
Jul 16 05:23:30.397799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.397779    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:30.399893 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.399870    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:23:30.404735 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.404709    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"]
Jul 16 05:23:30.503297 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.503256    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f148e-6775-4d55-a733-a5da62a855ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8142960b-x9rrv\" (UID: \"3b3f148e-6775-4d55-a733-a5da62a855ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:30.603999 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.603964    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f148e-6775-4d55-a733-a5da62a855ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8142960b-x9rrv\" (UID: \"3b3f148e-6775-4d55-a733-a5da62a855ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:30.604625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.604596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f148e-6775-4d55-a733-a5da62a855ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8142960b-x9rrv\" (UID: \"3b3f148e-6775-4d55-a733-a5da62a855ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:30.711270 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.711240    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:30.720599 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.720565    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:30.776020 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.775955    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:30.860758 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:30.860725    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"]
Jul 16 05:23:30.862039 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:23:30.862009    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3f148e_6775_4d55_a733_a5da62a855ed.slice/crio-952f0800df71ee4ac24fb45274f14150a2bf51bc48f6767cff70a370e0f7ecf2 WatchSource:0}: Error finding container 952f0800df71ee4ac24fb45274f14150a2bf51bc48f6767cff70a370e0f7ecf2: Status 404 returned error can't find the container with id 952f0800df71ee4ac24fb45274f14150a2bf51bc48f6767cff70a370e0f7ecf2
Jul 16 05:23:31.016508 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:31.016425    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" event={"ID":"3b3f148e-6775-4d55-a733-a5da62a855ed","Type":"ContainerStarted","Data":"952f0800df71ee4ac24fb45274f14150a2bf51bc48f6767cff70a370e0f7ecf2"}
Jul 16 05:23:32.023025 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:32.022938    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerID="efb5a3c4c8ec9d34941d9f106ed98eaf3de6b63dcecc7f5943e79a192db4aee1" exitCode=7
Jul 16 05:23:32.023384 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:32.023020    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" event={"ID":"3b3f148e-6775-4d55-a733-a5da62a855ed","Type":"ContainerDied","Data":"efb5a3c4c8ec9d34941d9f106ed98eaf3de6b63dcecc7f5943e79a192db4aee1"}
Jul 16 05:23:32.023384 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:32.023310    2565 scope.go:117] "RemoveContainer" containerID="efb5a3c4c8ec9d34941d9f106ed98eaf3de6b63dcecc7f5943e79a192db4aee1"
Jul 16 05:23:33.029908 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:33.029808    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerID="a23eeadb357032188fd87766564349bc2b5c9505a67aadc86fab8f3dcf59acdf" exitCode=7
Jul 16 05:23:33.029908 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:33.029860    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" event={"ID":"3b3f148e-6775-4d55-a733-a5da62a855ed","Type":"ContainerDied","Data":"a23eeadb357032188fd87766564349bc2b5c9505a67aadc86fab8f3dcf59acdf"}
Jul 16 05:23:33.030371 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:33.029920    2565 scope.go:117] "RemoveContainer" containerID="efb5a3c4c8ec9d34941d9f106ed98eaf3de6b63dcecc7f5943e79a192db4aee1"
Jul 16 05:23:33.030371 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:33.030132    2565 scope.go:117] "RemoveContainer" containerID="a23eeadb357032188fd87766564349bc2b5c9505a67aadc86fab8f3dcf59acdf"
Jul 16 05:23:33.030371 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:23:33.030357    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8142960b-x9rrv_odh-ai-gateway-infra(3b3f148e-6775-4d55-a733-a5da62a855ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed"
Jul 16 05:23:34.010478 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:34.010445    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:23:34.039191 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:34.039161    2565 scope.go:117] "RemoveContainer" containerID="a23eeadb357032188fd87766564349bc2b5c9505a67aadc86fab8f3dcf59acdf"
Jul 16 05:23:34.039565 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:23:34.039442    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8142960b-x9rrv_odh-ai-gateway-infra(3b3f148e-6775-4d55-a733-a5da62a855ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed"
Jul 16 05:23:34.109983 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:34.109955    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:41.760563 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:41.760530    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:23:45.645696 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:45.645661    2565 scope.go:117] "RemoveContainer" containerID="a23eeadb357032188fd87766564349bc2b5c9505a67aadc86fab8f3dcf59acdf"
Jul 16 05:23:46.109681 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:46.109647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" event={"ID":"3b3f148e-6775-4d55-a733-a5da62a855ed","Type":"ContainerStarted","Data":"00c377a155ce8b6f867ce4af26208768d57da46171e0af51bb6942aa7401ce01"}
Jul 16 05:23:46.126168 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:46.126120    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" podStartSLOduration=16.126104771 podStartE2EDuration="16.126104771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:23:46.123515023 +0000 UTC m=+2023.044097819" watchObservedRunningTime="2026-07-16 05:23:46.126104771 +0000 UTC m=+2023.046687566"
Jul 16 05:23:47.117904 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:47.117866    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerID="00c377a155ce8b6f867ce4af26208768d57da46171e0af51bb6942aa7401ce01" exitCode=0
Jul 16 05:23:47.118361 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:47.117942    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" event={"ID":"3b3f148e-6775-4d55-a733-a5da62a855ed","Type":"ContainerDied","Data":"00c377a155ce8b6f867ce4af26208768d57da46171e0af51bb6942aa7401ce01"}
Jul 16 05:23:47.118361 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:47.118002    2565 scope.go:117] "RemoveContainer" containerID="a23eeadb357032188fd87766564349bc2b5c9505a67aadc86fab8f3dcf59acdf"
Jul 16 05:23:47.134021 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:47.133994    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"]
Jul 16 05:23:48.265998 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:48.265969    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:48.367529 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:48.367496    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f148e-6775-4d55-a733-a5da62a855ed-maas-api-service-ca\") pod \"3b3f148e-6775-4d55-a733-a5da62a855ed\" (UID: \"3b3f148e-6775-4d55-a733-a5da62a855ed\") "
Jul 16 05:23:48.367861 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:48.367817    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b3f148e-6775-4d55-a733-a5da62a855ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b3f148e-6775-4d55-a733-a5da62a855ed" (UID: "3b3f148e-6775-4d55-a733-a5da62a855ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:23:48.468766 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:48.468682    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f148e-6775-4d55-a733-a5da62a855ed-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:23:49.131866 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:49.131820    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"
Jul 16 05:23:49.131866 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:49.131854    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv" event={"ID":"3b3f148e-6775-4d55-a733-a5da62a855ed","Type":"ContainerDied","Data":"952f0800df71ee4ac24fb45274f14150a2bf51bc48f6767cff70a370e0f7ecf2"}
Jul 16 05:23:49.132133 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:49.131905    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="952f0800df71ee4ac24fb45274f14150a2bf51bc48f6767cff70a370e0f7ecf2"
Jul 16 05:23:49.160025 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:49.159996    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"]
Jul 16 05:23:49.165630 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:49.165607    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-x9rrv"]
Jul 16 05:23:49.651905 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:23:49.651874    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" path="/var/lib/kubelet/pods/3b3f148e-6775-4d55-a733-a5da62a855ed/volumes"
Jul 16 05:24:20.435838 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.435801    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"]
Jul 16 05:24:20.436255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436208    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436219    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436226    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436231    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436254    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436430 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436260    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436430 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436325    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.436430 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.436336    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:24:20.439437 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.439420    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:20.441962 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.441926    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:24:20.445450 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.445422    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"]
Jul 16 05:24:20.481249 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.481214    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8142960b-wrq2z\" (UID: \"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:20.582400 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.582367    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8142960b-wrq2z\" (UID: \"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:20.583235 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.583208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8142960b-wrq2z\" (UID: \"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:20.751533 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.751453    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:20.886705 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:20.886678    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"]
Jul 16 05:24:20.892841 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:24:20.892795    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd42d5ff_5d86_4b4c_b2e3_4adb76381ab8.slice/crio-afdad3d9c43a1e50fdbca28e8928692af4a41d54b357df5f38007e75f1bec15c WatchSource:0}: Error finding container afdad3d9c43a1e50fdbca28e8928692af4a41d54b357df5f38007e75f1bec15c: Status 404 returned error can't find the container with id afdad3d9c43a1e50fdbca28e8928692af4a41d54b357df5f38007e75f1bec15c
Jul 16 05:24:21.304112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:21.304071    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z" event={"ID":"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8","Type":"ContainerStarted","Data":"f1c4ab8972cc0927714a46f84ce59afea510b9b617f89d3d802387fb9efa8ee3"}
Jul 16 05:24:21.304291 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:21.304141    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z" event={"ID":"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8","Type":"ContainerStarted","Data":"afdad3d9c43a1e50fdbca28e8928692af4a41d54b357df5f38007e75f1bec15c"}
Jul 16 05:24:21.318324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:21.318281    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z" podStartSLOduration=1.318267102 podStartE2EDuration="1.318267102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:24:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:24:21.317401822 +0000 UTC m=+2058.237984620" watchObservedRunningTime="2026-07-16 05:24:21.318267102 +0000 UTC m=+2058.238849897"
Jul 16 05:24:22.310299 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:22.310262    2565 generic.go:356] "Generic (PLEG): container finished" podID="dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8" containerID="f1c4ab8972cc0927714a46f84ce59afea510b9b617f89d3d802387fb9efa8ee3" exitCode=0
Jul 16 05:24:22.310667 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:22.310344    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z" event={"ID":"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8","Type":"ContainerDied","Data":"f1c4ab8972cc0927714a46f84ce59afea510b9b617f89d3d802387fb9efa8ee3"}
Jul 16 05:24:23.468184 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:23.468157    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:23.512119 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:23.512090    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8-maas-api-service-ca\") pod \"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8\" (UID: \"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8\") "
Jul 16 05:24:23.512562 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:23.512536    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8" (UID: "dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:24:23.613052 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:23.613017    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:24:24.322093 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:24.322063    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z"
Jul 16 05:24:24.322093 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:24.322090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8142960b-wrq2z" event={"ID":"dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8","Type":"ContainerDied","Data":"afdad3d9c43a1e50fdbca28e8928692af4a41d54b357df5f38007e75f1bec15c"}
Jul 16 05:24:24.322300 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:24.322135    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afdad3d9c43a1e50fdbca28e8928692af4a41d54b357df5f38007e75f1bec15c"
Jul 16 05:24:30.814642 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:30.814605    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"]
Jul 16 05:24:30.815147 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:30.814965    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292" podUID="24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" containerName="maas-api" containerID="cri-o://f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade" gracePeriod=30
Jul 16 05:24:30.838040 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:30.838011    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:31.069742 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.069721    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:24:31.194570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.194535    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-maas-api-tls\") pod \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") "
Jul 16 05:24:31.194717 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.194580    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-openshift-service-ca\") pod \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") "
Jul 16 05:24:31.194717 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.194609    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5xmn\" (UniqueName: \"kubernetes.io/projected/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-kube-api-access-m5xmn\") pod \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\" (UID: \"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe\") "
Jul 16 05:24:31.195046 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.195006    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" (UID: "24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:24:31.196916 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.196889    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-kube-api-access-m5xmn" (OuterVolumeSpecName: "kube-api-access-m5xmn") pod "24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" (UID: "24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe"). InnerVolumeSpecName "kube-api-access-m5xmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:24:31.197043 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.197020    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" (UID: "24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:24:31.295791 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.295751    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:24:31.295791 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.295785    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:24:31.295791 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.295800    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5xmn\" (UniqueName: \"kubernetes.io/projected/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe-kube-api-access-m5xmn\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:24:31.360872 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.360751    2565 generic.go:356] "Generic (PLEG): container finished" podID="24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" containerID="f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade" exitCode=0
Jul 16 05:24:31.360872 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.360799    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292" event={"ID":"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe","Type":"ContainerDied","Data":"f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade"}
Jul 16 05:24:31.360872 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.360808    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"
Jul 16 05:24:31.360872 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.360856    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292" event={"ID":"24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe","Type":"ContainerDied","Data":"5a2627ffd697c09b913ffbcf86f1edd2e76cadb747cc11672201b252e6d89a0e"}
Jul 16 05:24:31.360872 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.360879    2565 scope.go:117] "RemoveContainer" containerID="f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade"
Jul 16 05:24:31.370649 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.370632    2565 scope.go:117] "RemoveContainer" containerID="f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade"
Jul 16 05:24:31.370934 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:24:31.370917    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade\": container with ID starting with f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade not found: ID does not exist" containerID="f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade"
Jul 16 05:24:31.370988 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.370942    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade"} err="failed to get container status \"f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade\": rpc error: code = NotFound desc = could not find container \"f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade\": container with ID starting with f7a364770eec87ed13b8f00d043e0bc920f02217f085ec3041c48b302e1aeade not found: ID does not exist"
Jul 16 05:24:31.381793 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.381772    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"]
Jul 16 05:24:31.387506 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.387483    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8142960b-54b6bc64f8-j9292"]
Jul 16 05:24:31.651555 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:31.651460    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" path="/var/lib/kubelet/pods/24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe/volumes"
Jul 16 05:24:35.511060 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:35.511018    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:41.192020 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:41.191986    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:41.291995 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:41.291944    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:45.710974 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:45.710928    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:46.353301 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:46.353261    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:46.451232 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:46.451191    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:46.551929 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:46.551898    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:48.138537 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:48.138498    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:52.127353 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:52.127326    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:53.912694 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:53.912664    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:54.014784 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:54.014758    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:54.112627 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:54.112589    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:54.212630 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:54.212551    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:24:54.669579 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:24:54.669551    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:04.368059 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:04.367951    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:25:04.376261 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:04.371674    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:25:09.852374 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:09.852337    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:20.800641 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:20.800610    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:26.526995 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:26.526960    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:26.579416 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:26.579380    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:26.678120 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:26.678084    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:26.777315 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:26.777229    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:42.714387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:42.714353    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:48.323840 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:48.323791    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:48.375641 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:48.375614    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:48.473203 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:48.473168    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:48.572077 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:48.572048    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:59.134640 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:59.134543    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:25:59.936799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:25:59.936763    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:03.767179 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:03.767135    2565 scope.go:117] "RemoveContainer" containerID="7f19a1024ffdeabd1e50b5957297b6e0ed536c54639a52c96e9ae913d0aeeb06"
Jul 16 05:26:05.615692 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:05.615655    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:05.720545 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:05.720505    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:05.815108 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:05.815074    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:05.919364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:05.919279    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:06.783884 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:06.783823    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:06.881533 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:06.881491    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:06.985477 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:06.985447    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:08.760096 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:08.760057    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:14.410509 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:14.410476    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:14.510669 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:14.510634    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:15.110182 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:15.110147    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:20.716774 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:20.716735    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:20.817076 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:20.817037    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:21.020634 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:21.020561    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:21.284841 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:21.284736    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:21.348356 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:21.348325    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.261148 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.261116    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.341322 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.341275    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.348233 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.348197    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"]
Jul 16 05:26:24.348855 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.348807    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8" containerName="revoke-keys"
Jul 16 05:26:24.348855 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.348848    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8" containerName="revoke-keys"
Jul 16 05:26:24.348994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.348871    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" containerName="maas-api"
Jul 16 05:26:24.348994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.348880    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" containerName="maas-api"
Jul 16 05:26:24.349113 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.349023    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3f148e-6775-4d55-a733-a5da62a855ed" containerName="revoke-keys"
Jul 16 05:26:24.349113 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.349040    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c3bf6d-f8d1-4dbc-b292-b4f3ad4a0bbe" containerName="maas-api"
Jul 16 05:26:24.349113 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.349050    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8" containerName="revoke-keys"
Jul 16 05:26:24.352588 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.352569    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.355038 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.355020    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-44053aaf-serving-cert\""
Jul 16 05:26:24.362852 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.362799    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"]
Jul 16 05:26:24.406024 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.405994    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.440011 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.439983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14665aaa-9291-44ff-b22f-41146189e3d7-openshift-service-ca\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.440133 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.440013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48wj8\" (UniqueName: \"kubernetes.io/projected/14665aaa-9291-44ff-b22f-41146189e3d7-kube-api-access-48wj8\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.440133 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.440057    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14665aaa-9291-44ff-b22f-41146189e3d7-maas-api-tls\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.529464 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.529367    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.541471 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.541437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14665aaa-9291-44ff-b22f-41146189e3d7-openshift-service-ca\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.541611 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.541478    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48wj8\" (UniqueName: \"kubernetes.io/projected/14665aaa-9291-44ff-b22f-41146189e3d7-kube-api-access-48wj8\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.541611 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.541550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14665aaa-9291-44ff-b22f-41146189e3d7-maas-api-tls\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.542077 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.542059    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14665aaa-9291-44ff-b22f-41146189e3d7-openshift-service-ca\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.544204 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.544184    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14665aaa-9291-44ff-b22f-41146189e3d7-maas-api-tls\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.549204 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.549187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48wj8\" (UniqueName: \"kubernetes.io/projected/14665aaa-9291-44ff-b22f-41146189e3d7-kube-api-access-48wj8\") pod \"maas-api-e2e-mt-44053aaf-58f47645c6-7dm72\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.664868 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.664815    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:24.808048 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.808022    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"]
Jul 16 05:26:24.810431 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:26:24.810402    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14665aaa_9291_44ff_b22f_41146189e3d7.slice/crio-0937cb94620445ae0a506ea89f36c67ff56e35401c4993f6978561a440eb08d7 WatchSource:0}: Error finding container 0937cb94620445ae0a506ea89f36c67ff56e35401c4993f6978561a440eb08d7: Status 404 returned error can't find the container with id 0937cb94620445ae0a506ea89f36c67ff56e35401c4993f6978561a440eb08d7
Jul 16 05:26:24.811898 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.811881    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 05:26:24.822476 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.822445    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.923852 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.923795    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:24.975248 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:24.975215    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72" event={"ID":"14665aaa-9291-44ff-b22f-41146189e3d7","Type":"ContainerStarted","Data":"0937cb94620445ae0a506ea89f36c67ff56e35401c4993f6978561a440eb08d7"}
Jul 16 05:26:25.992329 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:25.992274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72" event={"ID":"14665aaa-9291-44ff-b22f-41146189e3d7","Type":"ContainerStarted","Data":"cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7"}
Jul 16 05:26:25.992795 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:25.992378    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:26.008418 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:26.008367    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72" podStartSLOduration=1.4330822429999999 podStartE2EDuration="2.00835225s" totalImagesPullingTime="575.270007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:26:26.00660756 +0000 UTC m=+2182.927190354" watchObservedRunningTime="2026-07-16 05:26:26.00835225 +0000 UTC m=+2182.928935046"
Jul 16 05:26:28.835351 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:28.835312    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:29.559894 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:29.559858    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:29.678909 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:29.678876    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:30.045847 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:30.045785    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:32.002879 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:32.002849    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:35.847257 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:35.847215    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:35.947576 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:35.947521    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:36.262217 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.262138    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"]
Jul 16 05:26:36.265892 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.265876    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:36.268287 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.268263    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:26:36.272311 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.272286    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"]
Jul 16 05:26:36.354260 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.354222    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f365a36-f021-425e-bd07-d93b221a16ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-44053aaf-wp922\" (UID: \"7f365a36-f021-425e-bd07-d93b221a16ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:36.455474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.455437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f365a36-f021-425e-bd07-d93b221a16ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-44053aaf-wp922\" (UID: \"7f365a36-f021-425e-bd07-d93b221a16ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:36.456079 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.456060    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f365a36-f021-425e-bd07-d93b221a16ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-44053aaf-wp922\" (UID: \"7f365a36-f021-425e-bd07-d93b221a16ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:36.579357 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.579320    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:36.723971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:36.723934    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"]
Jul 16 05:26:36.725445 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:26:36.725410    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f365a36_f021_425e_bd07_d93b221a16ad.slice/crio-392eac3710644121f7a676ed388781f997fc893f7d331c27ef06d4447587a08b WatchSource:0}: Error finding container 392eac3710644121f7a676ed388781f997fc893f7d331c27ef06d4447587a08b: Status 404 returned error can't find the container with id 392eac3710644121f7a676ed388781f997fc893f7d331c27ef06d4447587a08b
Jul 16 05:26:37.051976 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:37.051941    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922" event={"ID":"7f365a36-f021-425e-bd07-d93b221a16ad","Type":"ContainerStarted","Data":"cf54d8ffcd5599322487cbd5904c646f798c590f9b9ebe57e3890a38d5863568"}
Jul 16 05:26:37.052365 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:37.052008    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922" event={"ID":"7f365a36-f021-425e-bd07-d93b221a16ad","Type":"ContainerStarted","Data":"392eac3710644121f7a676ed388781f997fc893f7d331c27ef06d4447587a08b"}
Jul 16 05:26:37.068874 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:37.068778    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922" podStartSLOduration=1.068757851 podStartE2EDuration="1.068757851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:26:37.064177365 +0000 UTC m=+2193.984760161" watchObservedRunningTime="2026-07-16 05:26:37.068757851 +0000 UTC m=+2193.989340647"
Jul 16 05:26:38.061812 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:38.061272    2565 generic.go:356] "Generic (PLEG): container finished" podID="7f365a36-f021-425e-bd07-d93b221a16ad" containerID="cf54d8ffcd5599322487cbd5904c646f798c590f9b9ebe57e3890a38d5863568" exitCode=0
Jul 16 05:26:38.061812 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:38.061437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922" event={"ID":"7f365a36-f021-425e-bd07-d93b221a16ad","Type":"ContainerDied","Data":"cf54d8ffcd5599322487cbd5904c646f798c590f9b9ebe57e3890a38d5863568"}
Jul 16 05:26:39.207296 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:39.207270    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:39.282770 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:39.282735    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f365a36-f021-425e-bd07-d93b221a16ad-maas-api-service-ca\") pod \"7f365a36-f021-425e-bd07-d93b221a16ad\" (UID: \"7f365a36-f021-425e-bd07-d93b221a16ad\") "
Jul 16 05:26:39.283120 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:39.283096    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f365a36-f021-425e-bd07-d93b221a16ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f365a36-f021-425e-bd07-d93b221a16ad" (UID: "7f365a36-f021-425e-bd07-d93b221a16ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:26:39.383900 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:39.383862    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f365a36-f021-425e-bd07-d93b221a16ad-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:26:40.073624 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:40.073597    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922"
Jul 16 05:26:40.073798 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:40.073643    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-44053aaf-wp922" event={"ID":"7f365a36-f021-425e-bd07-d93b221a16ad","Type":"ContainerDied","Data":"392eac3710644121f7a676ed388781f997fc893f7d331c27ef06d4447587a08b"}
Jul 16 05:26:40.073798 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:40.073696    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="392eac3710644121f7a676ed388781f997fc893f7d331c27ef06d4447587a08b"
Jul 16 05:26:46.629925 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.629882    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"]
Jul 16 05:26:46.630525 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.630115    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72" podUID="14665aaa-9291-44ff-b22f-41146189e3d7" containerName="maas-api" containerID="cri-o://cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7" gracePeriod=30
Jul 16 05:26:46.654407 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.654380    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:46.881216 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.881150    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:46.955659 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.955622    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14665aaa-9291-44ff-b22f-41146189e3d7-openshift-service-ca\") pod \"14665aaa-9291-44ff-b22f-41146189e3d7\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") "
Jul 16 05:26:46.955659 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.955660    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48wj8\" (UniqueName: \"kubernetes.io/projected/14665aaa-9291-44ff-b22f-41146189e3d7-kube-api-access-48wj8\") pod \"14665aaa-9291-44ff-b22f-41146189e3d7\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") "
Jul 16 05:26:46.955932 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.955681    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14665aaa-9291-44ff-b22f-41146189e3d7-maas-api-tls\") pod \"14665aaa-9291-44ff-b22f-41146189e3d7\" (UID: \"14665aaa-9291-44ff-b22f-41146189e3d7\") "
Jul 16 05:26:46.956035 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.956008    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14665aaa-9291-44ff-b22f-41146189e3d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14665aaa-9291-44ff-b22f-41146189e3d7" (UID: "14665aaa-9291-44ff-b22f-41146189e3d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:26:46.957809 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.957786    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14665aaa-9291-44ff-b22f-41146189e3d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14665aaa-9291-44ff-b22f-41146189e3d7" (UID: "14665aaa-9291-44ff-b22f-41146189e3d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:26:46.957899 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:46.957817    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14665aaa-9291-44ff-b22f-41146189e3d7-kube-api-access-48wj8" (OuterVolumeSpecName: "kube-api-access-48wj8") pod "14665aaa-9291-44ff-b22f-41146189e3d7" (UID: "14665aaa-9291-44ff-b22f-41146189e3d7"). InnerVolumeSpecName "kube-api-access-48wj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:26:47.056856 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.056792    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14665aaa-9291-44ff-b22f-41146189e3d7-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:26:47.056856 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.056822    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48wj8\" (UniqueName: \"kubernetes.io/projected/14665aaa-9291-44ff-b22f-41146189e3d7-kube-api-access-48wj8\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:26:47.057048 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.056867    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14665aaa-9291-44ff-b22f-41146189e3d7-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:26:47.111185 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.111148    2565 generic.go:356] "Generic (PLEG): container finished" podID="14665aaa-9291-44ff-b22f-41146189e3d7" containerID="cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7" exitCode=0
Jul 16 05:26:47.111372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.111198    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"
Jul 16 05:26:47.111372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.111244    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72" event={"ID":"14665aaa-9291-44ff-b22f-41146189e3d7","Type":"ContainerDied","Data":"cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7"}
Jul 16 05:26:47.111372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.111312    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72" event={"ID":"14665aaa-9291-44ff-b22f-41146189e3d7","Type":"ContainerDied","Data":"0937cb94620445ae0a506ea89f36c67ff56e35401c4993f6978561a440eb08d7"}
Jul 16 05:26:47.111372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.111329    2565 scope.go:117] "RemoveContainer" containerID="cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7"
Jul 16 05:26:47.120653 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.120635    2565 scope.go:117] "RemoveContainer" containerID="cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7"
Jul 16 05:26:47.121041 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:26:47.121009    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7\": container with ID starting with cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7 not found: ID does not exist" containerID="cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7"
Jul 16 05:26:47.121130 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.121051    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7"} err="failed to get container status \"cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7\": rpc error: code = NotFound desc = could not find container \"cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7\": container with ID starting with cbb9b441b3a87f0a45cf1e9f9b89b7918ebc280e743bfcd7f07912838b3be4b7 not found: ID does not exist"
Jul 16 05:26:47.131689 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.131635    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"]
Jul 16 05:26:47.135519 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.135481    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-44053aaf-58f47645c6-7dm72"]
Jul 16 05:26:47.651236 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:47.651204    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14665aaa-9291-44ff-b22f-41146189e3d7" path="/var/lib/kubelet/pods/14665aaa-9291-44ff-b22f-41146189e3d7/volumes"
Jul 16 05:26:51.438767 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:51.438737    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:56.981972 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:56.981923    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:26:57.079452 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:26:57.079419    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:02.218732 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:02.218694    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:03.827445 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:03.827398    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:08.363841 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:08.363806    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:08.426193 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:08.426165    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:08.524477 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:08.524444    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:08.632190 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:08.632076    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:08.803520 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:08.803480    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:14.441008 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:14.440968    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:14.540779 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:14.540743    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:16.556375 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:16.556337    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:16.658643 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:16.658613    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:21.368507 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:21.368468    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:21.693681 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:21.693607    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:22.140761 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:22.140733    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:24.242364 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.242326    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:24.322543 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.322500    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:24.324110 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324082    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"]
Jul 16 05:27:24.324631 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324610    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14665aaa-9291-44ff-b22f-41146189e3d7" containerName="maas-api"
Jul 16 05:27:24.324738 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324633    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="14665aaa-9291-44ff-b22f-41146189e3d7" containerName="maas-api"
Jul 16 05:27:24.324738 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324656    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f365a36-f021-425e-bd07-d93b221a16ad" containerName="revoke-keys"
Jul 16 05:27:24.324738 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324664    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f365a36-f021-425e-bd07-d93b221a16ad" containerName="revoke-keys"
Jul 16 05:27:24.324933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324775    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="14665aaa-9291-44ff-b22f-41146189e3d7" containerName="maas-api"
Jul 16 05:27:24.324933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.324790    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f365a36-f021-425e-bd07-d93b221a16ad" containerName="revoke-keys"
Jul 16 05:27:24.328253 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.328234    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.330971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.330951    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-18c409ba-serving-cert\""
Jul 16 05:27:24.337447 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.337423    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"]
Jul 16 05:27:24.398875 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.398841    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:24.411561 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.411537    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-openshift-service-ca\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.411644 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.411566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55qwt\" (UniqueName: \"kubernetes.io/projected/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-kube-api-access-55qwt\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.411687 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.411657    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.512263 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.512188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.512263 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.512255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-openshift-service-ca\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.512420 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.512274    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55qwt\" (UniqueName: \"kubernetes.io/projected/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-kube-api-access-55qwt\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.512420 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:27:24.512339    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-serving-cert: secret "maas-api-e2e-mt-18c409ba-serving-cert" not found
Jul 16 05:27:24.512489 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:27:24.512424    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls podName:e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c nodeName:}" failed. No retries permitted until 2026-07-16 05:27:25.012405285 +0000 UTC m=+2241.932988059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls") pod "maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" (UID: "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c") : secret "maas-api-e2e-mt-18c409ba-serving-cert" not found
Jul 16 05:27:24.512891 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.512868    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-openshift-service-ca\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.522778 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.522752    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55qwt\" (UniqueName: \"kubernetes.io/projected/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-kube-api-access-55qwt\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:24.702619 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.702543    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:24.893892 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:24.893860    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:25.010818 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:25.010784    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:25.017238 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:25.017212    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:25.019694 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:25.019671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls\") pod \"maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:25.240348 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:25.240268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:25.383729 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:25.383688    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"]
Jul 16 05:27:25.384463 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:27:25.384436    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a94f32_8ad4_4c3d_831d_e1c8c6d6645c.slice/crio-631415d96d08db181e156ee483a60e7381774eea0f88f97b56374318a1a7d527 WatchSource:0}: Error finding container 631415d96d08db181e156ee483a60e7381774eea0f88f97b56374318a1a7d527: Status 404 returned error can't find the container with id 631415d96d08db181e156ee483a60e7381774eea0f88f97b56374318a1a7d527
Jul 16 05:27:26.318964 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:26.318933    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" event={"ID":"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c","Type":"ContainerStarted","Data":"d282a901f8897dcf13f744ae21c705efc7ed9c8902e2556f915c0be9e31e912f"}
Jul 16 05:27:26.319137 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:26.318983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" event={"ID":"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c","Type":"ContainerStarted","Data":"631415d96d08db181e156ee483a60e7381774eea0f88f97b56374318a1a7d527"}
Jul 16 05:27:26.319137 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:26.319057    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:26.338152 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:26.338109    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" podStartSLOduration=1.941645427 podStartE2EDuration="2.33809099s" totalImagesPullingTime="396.445563ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:27:26.335202293 +0000 UTC m=+2243.255785089" watchObservedRunningTime="2026-07-16 05:27:26.33809099 +0000 UTC m=+2243.258673811"
Jul 16 05:27:28.777517 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:28.777487    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:32.329492 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:32.329460    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:27:35.093452 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:35.093421    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:27:40.062506 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:27:40.062466    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:28:03.805712 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:03.805677    2565 scope.go:117] "RemoveContainer" containerID="913ba4a65a99d2ded4ac95cbe843871ff16c53f835911dd65c075e0501e539f3"
Jul 16 05:28:12.922255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:12.922219    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"]
Jul 16 05:28:12.925815 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:12.925796    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:12.928148 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:12.928132    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:28:12.931297 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:12.931274    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"]
Jul 16 05:28:13.056538 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:13.056501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3550f49c-0f0f-408d-814c-1f0e026faa48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h\" (UID: \"3550f49c-0f0f-408d-814c-1f0e026faa48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:13.157498 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:13.157460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3550f49c-0f0f-408d-814c-1f0e026faa48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h\" (UID: \"3550f49c-0f0f-408d-814c-1f0e026faa48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:13.158102 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:13.158082    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3550f49c-0f0f-408d-814c-1f0e026faa48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h\" (UID: \"3550f49c-0f0f-408d-814c-1f0e026faa48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:13.237369 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:13.237293    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:13.377394 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:13.377360    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"]
Jul 16 05:28:13.378571 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:28:13.378534    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3550f49c_0f0f_408d_814c_1f0e026faa48.slice/crio-0a3490512cec33e07853d87ea337a828612377e585343990d2ed5729a1fcab21 WatchSource:0}: Error finding container 0a3490512cec33e07853d87ea337a828612377e585343990d2ed5729a1fcab21: Status 404 returned error can't find the container with id 0a3490512cec33e07853d87ea337a828612377e585343990d2ed5729a1fcab21
Jul 16 05:28:13.575625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:13.575595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h" event={"ID":"3550f49c-0f0f-408d-814c-1f0e026faa48","Type":"ContainerStarted","Data":"0a3490512cec33e07853d87ea337a828612377e585343990d2ed5729a1fcab21"}
Jul 16 05:28:14.582360 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:14.582328    2565 generic.go:356] "Generic (PLEG): container finished" podID="3550f49c-0f0f-408d-814c-1f0e026faa48" containerID="13098d7e9b81017a397cf173f7127602471f97a10fcd2747bb2151ad0f6f6841" exitCode=0
Jul 16 05:28:14.582749 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:14.582422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h" event={"ID":"3550f49c-0f0f-408d-814c-1f0e026faa48","Type":"ContainerDied","Data":"13098d7e9b81017a397cf173f7127602471f97a10fcd2747bb2151ad0f6f6841"}
Jul 16 05:28:15.732614 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:15.732589    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:15.883524 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:15.883433    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3550f49c-0f0f-408d-814c-1f0e026faa48-maas-api-service-ca\") pod \"3550f49c-0f0f-408d-814c-1f0e026faa48\" (UID: \"3550f49c-0f0f-408d-814c-1f0e026faa48\") "
Jul 16 05:28:15.883767 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:15.883747    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3550f49c-0f0f-408d-814c-1f0e026faa48-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3550f49c-0f0f-408d-814c-1f0e026faa48" (UID: "3550f49c-0f0f-408d-814c-1f0e026faa48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:28:15.984488 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:15.984456    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3550f49c-0f0f-408d-814c-1f0e026faa48-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:28:16.595008 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:16.594968    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h" event={"ID":"3550f49c-0f0f-408d-814c-1f0e026faa48","Type":"ContainerDied","Data":"0a3490512cec33e07853d87ea337a828612377e585343990d2ed5729a1fcab21"}
Jul 16 05:28:16.595008 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:16.595000    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h"
Jul 16 05:28:16.595218 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:16.595032    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a3490512cec33e07853d87ea337a828612377e585343990d2ed5729a1fcab21"
Jul 16 05:28:23.446321 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.446287    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"]
Jul 16 05:28:23.446894 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.446516    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" podUID="e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" containerName="maas-api" containerID="cri-o://d282a901f8897dcf13f744ae21c705efc7ed9c8902e2556f915c0be9e31e912f" gracePeriod=30
Jul 16 05:28:23.471877 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.471848    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:28:23.637787 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.637757    2565 generic.go:356] "Generic (PLEG): container finished" podID="e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" containerID="d282a901f8897dcf13f744ae21c705efc7ed9c8902e2556f915c0be9e31e912f" exitCode=0
Jul 16 05:28:23.637961 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.637815    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" event={"ID":"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c","Type":"ContainerDied","Data":"d282a901f8897dcf13f744ae21c705efc7ed9c8902e2556f915c0be9e31e912f"}
Jul 16 05:28:23.710203 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.710134    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:28:23.758881 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.758819    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls\") pod \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") "
Jul 16 05:28:23.759033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.758921    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55qwt\" (UniqueName: \"kubernetes.io/projected/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-kube-api-access-55qwt\") pod \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") "
Jul 16 05:28:23.759033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.758942    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-openshift-service-ca\") pod \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\" (UID: \"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c\") "
Jul 16 05:28:23.759375 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.759351    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" (UID: "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:28:23.760904 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.760883    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" (UID: "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:28:23.761187 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.761174    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-kube-api-access-55qwt" (OuterVolumeSpecName: "kube-api-access-55qwt") pod "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" (UID: "e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c"). InnerVolumeSpecName "kube-api-access-55qwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:28:23.859631 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.859598    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:28:23.859631 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.859627    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55qwt\" (UniqueName: \"kubernetes.io/projected/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-kube-api-access-55qwt\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:28:23.859810 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:23.859640    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:28:24.644776 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:24.644741    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl" event={"ID":"e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c","Type":"ContainerDied","Data":"631415d96d08db181e156ee483a60e7381774eea0f88f97b56374318a1a7d527"}
Jul 16 05:28:24.644776 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:24.644775    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"
Jul 16 05:28:24.645305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:24.644793    2565 scope.go:117] "RemoveContainer" containerID="d282a901f8897dcf13f744ae21c705efc7ed9c8902e2556f915c0be9e31e912f"
Jul 16 05:28:24.665972 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:24.665950    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"]
Jul 16 05:28:24.669414 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:24.669395    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18c409ba-68b5f79cb4-9d4fl"]
Jul 16 05:28:25.650929 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:25.650898    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" path="/var/lib/kubelet/pods/e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c/volumes"
Jul 16 05:28:28.002773 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:28.002743    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:28:33.027885 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:28:33.027851    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:03.826546 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:03.826471    2565 scope.go:117] "RemoveContainer" containerID="74ab8cde932eefe90cb5910ee88eb8cae1b657f89b203003bdaa5dc66dfb3b04"
Jul 16 05:29:09.135358 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:09.135320    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:10.038879 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:10.038840    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:15.437082 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:15.437046    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:29.391007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:29.390965    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:29.454082 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:29.454036    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:40.365903 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:40.365852    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:40.614612 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:40.614582    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:41.744961 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:41.744920    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:41.839032 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:41.838994    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:41.939953 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:41.939918    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:42.791319 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:42.791279    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:42.889527 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:42.889493    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:42.993683 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:42.993645    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:44.481601 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:44.481556    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:50.080373 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:50.080344    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:50.178520 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:50.178477    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:50.794083 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:50.794044    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:56.418311 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:56.418273    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:56.477015 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:56.476984    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:56.566448 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:56.566401    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:29:56.663718 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:29:56.663687    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:00.138313 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138272    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"]
Jul 16 05:30:00.138797 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138781    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" containerName="maas-api"
Jul 16 05:30:00.138797 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138798    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" containerName="maas-api"
Jul 16 05:30:00.138912 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138812    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3550f49c-0f0f-408d-814c-1f0e026faa48" containerName="revoke-keys"
Jul 16 05:30:00.138912 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138818    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3550f49c-0f0f-408d-814c-1f0e026faa48" containerName="revoke-keys"
Jul 16 05:30:00.138978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138913    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3550f49c-0f0f-408d-814c-1f0e026faa48" containerName="revoke-keys"
Jul 16 05:30:00.138978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.138924    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a94f32-8ad4-4c3d-831d-e1c8c6d6645c" containerName="maas-api"
Jul 16 05:30:00.142255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.142240    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"
Jul 16 05:30:00.144894 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.144862    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:30:00.146780 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.146754    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"]
Jul 16 05:30:00.154970 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.154946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"
Jul 16 05:30:00.300197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:00.300167    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"]
Jul 16 05:30:00.301935 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:30:00.301908    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a78444b_6b99_4c63_a978_84c178993d04.slice/crio-2aa993c62eae8af2b3dc9f6ab52e1bb220ea90784be15c3c74565b379c3f37a5 WatchSource:0}: Error finding container 2aa993c62eae8af2b3dc9f6ab52e1bb220ea90784be15c3c74565b379c3f37a5: Status 404 returned error can't find the container with id 2aa993c62eae8af2b3dc9f6ab52e1bb220ea90784be15c3c74565b379c3f37a5
Jul 16 05:30:01.162966 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:01.162925    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h" event={"ID":"1a78444b-6b99-4c63-a978-84c178993d04","Type":"ContainerStarted","Data":"a1c5a17b6067e6d56486345a55beb9f7058b35f5d603a0f69c059ae2b0a4c1a6"}
Jul 16 05:30:01.163349 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:01.162992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h" event={"ID":"1a78444b-6b99-4c63-a978-84c178993d04","Type":"ContainerStarted","Data":"2aa993c62eae8af2b3dc9f6ab52e1bb220ea90784be15c3c74565b379c3f37a5"}
Jul 16 05:30:01.176611 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:01.176561    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h" podStartSLOduration=1.176544458 podStartE2EDuration="1.176544458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:30:01.17531754 +0000 UTC m=+2398.095900332" watchObservedRunningTime="2026-07-16 05:30:01.176544458 +0000 UTC m=+2398.097127309"
Jul 16 05:30:02.169994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:02.169958    2565 generic.go:356] "Generic (PLEG): container finished" podID="1a78444b-6b99-4c63-a978-84c178993d04" containerID="a1c5a17b6067e6d56486345a55beb9f7058b35f5d603a0f69c059ae2b0a4c1a6" exitCode=0
Jul 16 05:30:02.170395 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:02.170038    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h" event={"ID":"1a78444b-6b99-4c63-a978-84c178993d04","Type":"ContainerDied","Data":"a1c5a17b6067e6d56486345a55beb9f7058b35f5d603a0f69c059ae2b0a4c1a6"}
Jul 16 05:30:03.313585 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:03.313555    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"
Jul 16 05:30:03.854682 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:03.854650    2565 scope.go:117] "RemoveContainer" containerID="00c377a155ce8b6f867ce4af26208768d57da46171e0af51bb6942aa7401ce01"
Jul 16 05:30:04.182492 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:04.182452    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h"
Jul 16 05:30:04.182492 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:04.182472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736330-8qj8h" event={"ID":"1a78444b-6b99-4c63-a978-84c178993d04","Type":"ContainerDied","Data":"2aa993c62eae8af2b3dc9f6ab52e1bb220ea90784be15c3c74565b379c3f37a5"}
Jul 16 05:30:04.182702 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:04.182547    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aa993c62eae8af2b3dc9f6ab52e1bb220ea90784be15c3c74565b379c3f37a5"
Jul 16 05:30:04.399697 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:04.399575    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:30:04.417109 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:04.404177    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:30:07.257212 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:07.257180    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:07.651079 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:07.651043    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:28.711809 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:28.711722    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:34.372897 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:34.372855    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:34.473009 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:34.472977    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:34.781002 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:34.780915    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:35.104052 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:35.104018    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:35.202334 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:35.202299    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:35.899466 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:35.899434    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:38.090372 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.090333    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:38.162128 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.162089    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"]
Jul 16 05:30:38.162512 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.162500    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a78444b-6b99-4c63-a978-84c178993d04" containerName="cleanup"
Jul 16 05:30:38.162570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.162514    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a78444b-6b99-4c63-a978-84c178993d04" containerName="cleanup"
Jul 16 05:30:38.162607 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.162592    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a78444b-6b99-4c63-a978-84c178993d04" containerName="cleanup"
Jul 16 05:30:38.166080 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.166060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.168508 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.168488    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a956af-serving-cert\""
Jul 16 05:30:38.171033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.171007    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:38.178476 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.178447    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"]
Jul 16 05:30:38.189066 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.189027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.189203 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.189097    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.189203 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.189121    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlrk\" (UniqueName: \"kubernetes.io/projected/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-kube-api-access-2nlrk\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.237465 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.237420    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:38.289858 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.289797    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.290075 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.289870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.290075 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.289901    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlrk\" (UniqueName: \"kubernetes.io/projected/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-kube-api-access-2nlrk\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.290075 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:30:38.289970    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-serving-cert: secret "maas-api-e2e-inf-a-a956af-serving-cert" not found
Jul 16 05:30:38.290075 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:30:38.290059    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls podName:61ccc2e3-707a-4d31-9d57-616a1da1ea1a nodeName:}" failed. No retries permitted until 2026-07-16 05:30:38.79003765 +0000 UTC m=+2435.710620422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls") pod "maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" (UID: "61ccc2e3-707a-4d31-9d57-616a1da1ea1a") : secret "maas-api-e2e-inf-a-a956af-serving-cert" not found
Jul 16 05:30:38.290598 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.290579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.299265 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.299234    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlrk\" (UniqueName: \"kubernetes.io/projected/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-kube-api-access-2nlrk\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.395107 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.395003    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:38.730821 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.730723    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:38.794026 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.793990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.796625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.796597    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls\") pod \"maas-api-e2e-inf-a-a956af-55cf585f6b-q775n\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:38.868738 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:38.868704    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:39.077566 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:39.077529    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:39.228202 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:39.228168    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"]
Jul 16 05:30:39.230208 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:30:39.230184    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ccc2e3_707a_4d31_9d57_616a1da1ea1a.slice/crio-e8a51c5a19693e8a390e0f7e2947400591b483c3aa05affcb3684fa1d60f9ef7 WatchSource:0}: Error finding container e8a51c5a19693e8a390e0f7e2947400591b483c3aa05affcb3684fa1d60f9ef7: Status 404 returned error can't find the container with id e8a51c5a19693e8a390e0f7e2947400591b483c3aa05affcb3684fa1d60f9ef7
Jul 16 05:30:39.375039 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:39.374949    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" event={"ID":"61ccc2e3-707a-4d31-9d57-616a1da1ea1a","Type":"ContainerStarted","Data":"e8a51c5a19693e8a390e0f7e2947400591b483c3aa05affcb3684fa1d60f9ef7"}
Jul 16 05:30:40.382140 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:40.382109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" event={"ID":"61ccc2e3-707a-4d31-9d57-616a1da1ea1a","Type":"ContainerStarted","Data":"5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1"}
Jul 16 05:30:40.382622 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:40.382292    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:40.399627 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:40.399574    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" podStartSLOduration=1.727116427 podStartE2EDuration="2.39956025s" totalImagesPullingTime="672.443823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:30:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:30:40.397041285 +0000 UTC m=+2437.317624078" watchObservedRunningTime="2026-07-16 05:30:40.39956025 +0000 UTC m=+2437.320143045"
Jul 16 05:30:42.660225 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:42.660196    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:43.070697 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:43.070654    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:43.406509 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:43.406418    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:43.588947 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:43.588906    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:46.167884 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.167819    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:46.267209 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.267176    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:46.345400 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.345363    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"]
Jul 16 05:30:46.349997 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.349960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.352454 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.352420    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-309054-serving-cert\""
Jul 16 05:30:46.358511 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.358488    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"]
Jul 16 05:30:46.386538 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.386500    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:46.394393 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.394368    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:30:46.469027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.468931    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.469027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.468992    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26p5c\" (UniqueName: \"kubernetes.io/projected/af9d8124-64b2-4aa3-9520-226c3988f375-kube-api-access-26p5c\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.469223 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.469112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af9d8124-64b2-4aa3-9520-226c3988f375-openshift-service-ca\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.570320 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.570280    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af9d8124-64b2-4aa3-9520-226c3988f375-openshift-service-ca\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.570481 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.570371    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.570481 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.570392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26p5c\" (UniqueName: \"kubernetes.io/projected/af9d8124-64b2-4aa3-9520-226c3988f375-kube-api-access-26p5c\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.570583 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:30:46.570525    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-serving-cert: secret "maas-api-e2e-inf-b-309054-serving-cert" not found
Jul 16 05:30:46.570634 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:30:46.570620    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls podName:af9d8124-64b2-4aa3-9520-226c3988f375 nodeName:}" failed. No retries permitted until 2026-07-16 05:30:47.070596567 +0000 UTC m=+2443.991179349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls") pod "maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" (UID: "af9d8124-64b2-4aa3-9520-226c3988f375") : secret "maas-api-e2e-inf-b-309054-serving-cert" not found
Jul 16 05:30:46.571089 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.571072    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af9d8124-64b2-4aa3-9520-226c3988f375-openshift-service-ca\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.580435 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.580398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26p5c\" (UniqueName: \"kubernetes.io/projected/af9d8124-64b2-4aa3-9520-226c3988f375-kube-api-access-26p5c\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:46.830255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.830228    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:46.931257 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:46.931220    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:47.075335 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:47.075295    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:47.075508 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:30:47.075454    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-serving-cert: secret "maas-api-e2e-inf-b-309054-serving-cert" not found
Jul 16 05:30:47.075554 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:30:47.075527    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls podName:af9d8124-64b2-4aa3-9520-226c3988f375 nodeName:}" failed. No retries permitted until 2026-07-16 05:30:48.075510833 +0000 UTC m=+2444.996093605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls") pod "maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" (UID: "af9d8124-64b2-4aa3-9520-226c3988f375") : secret "maas-api-e2e-inf-b-309054-serving-cert" not found
Jul 16 05:30:47.276665 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:47.276568    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:48.086043 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:48.085997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:48.088961 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:48.088929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") pod \"maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:48.165536 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:48.165500    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:48.313716 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:48.313688    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"]
Jul 16 05:30:48.316596 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:30:48.316563    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9d8124_64b2_4aa3_9520_226c3988f375.slice/crio-9f1d59d70e06125fc7ee48d809ab8415c2176258ff9a433b247ad1e41f071a28 WatchSource:0}: Error finding container 9f1d59d70e06125fc7ee48d809ab8415c2176258ff9a433b247ad1e41f071a28: Status 404 returned error can't find the container with id 9f1d59d70e06125fc7ee48d809ab8415c2176258ff9a433b247ad1e41f071a28
Jul 16 05:30:48.429178 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:48.429087    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" event={"ID":"af9d8124-64b2-4aa3-9520-226c3988f375","Type":"ContainerStarted","Data":"9f1d59d70e06125fc7ee48d809ab8415c2176258ff9a433b247ad1e41f071a28"}
Jul 16 05:30:49.436065 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:49.436027    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" event={"ID":"af9d8124-64b2-4aa3-9520-226c3988f375","Type":"ContainerStarted","Data":"a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5"}
Jul 16 05:30:49.436469 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:49.436122    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:49.451789 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:49.451743    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" podStartSLOduration=3.145218943 podStartE2EDuration="3.451727185s" totalImagesPullingTime="306.508242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:30:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:30:49.450742852 +0000 UTC m=+2446.371325651" watchObservedRunningTime="2026-07-16 05:30:49.451727185 +0000 UTC m=+2446.372309979"
Jul 16 05:30:50.251144 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:50.251110    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:50.773137 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:50.773099    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:53.089938 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:53.089877    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:53.191255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:53.191209    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:55.446933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:55.446903    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:30:59.637206 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:59.637165    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:30:59.737631 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:30:59.737594    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:03.329261 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:03.329220    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:03.383268 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:03.383228    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:07.651257 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:07.651223    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:07.729027 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:07.728992    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:25.764740 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:25.764704    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:44.612414 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:44.612378    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:45.574465 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.574431    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:45.692434 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.692398    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"]
Jul 16 05:31:45.696418 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.696394    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:45.698692 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.698671    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:31:45.702809 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.702783    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"]
Jul 16 05:31:45.761971 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.761930    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e0a71f9-3bdf-42b9-ba6e-94d5d109b483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a956af-nrtck\" (UID: \"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:45.862508 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.862417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e0a71f9-3bdf-42b9-ba6e-94d5d109b483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a956af-nrtck\" (UID: \"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:45.863106 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:45.863088    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e0a71f9-3bdf-42b9-ba6e-94d5d109b483-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a956af-nrtck\" (UID: \"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:46.011939 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.011903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:46.151984 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.151958    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"]
Jul 16 05:31:46.154138 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:31:46.154112    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e0a71f9_3bdf_42b9_ba6e_94d5d109b483.slice/crio-c44a62ad8b01740816e9b04103f5dcca861301070ab899b4fd97bb4487a0f882 WatchSource:0}: Error finding container c44a62ad8b01740816e9b04103f5dcca861301070ab899b4fd97bb4487a0f882: Status 404 returned error can't find the container with id c44a62ad8b01740816e9b04103f5dcca861301070ab899b4fd97bb4487a0f882
Jul 16 05:31:46.284958 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.284923    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:46.386508 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.386472    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:46.750346 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.750246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck" event={"ID":"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483","Type":"ContainerStarted","Data":"54c45890f4aa6e5463350ad49024cfeebb814b6486ce2b9a18ca38103a7bfdb6"}
Jul 16 05:31:46.750346 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.750308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck" event={"ID":"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483","Type":"ContainerStarted","Data":"c44a62ad8b01740816e9b04103f5dcca861301070ab899b4fd97bb4487a0f882"}
Jul 16 05:31:46.765033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:46.764982    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck" podStartSLOduration=1.764966366 podStartE2EDuration="1.764966366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:31:46.762511222 +0000 UTC m=+2503.683094018" watchObservedRunningTime="2026-07-16 05:31:46.764966366 +0000 UTC m=+2503.685549180"
Jul 16 05:31:47.765113 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:47.765074    2565 generic.go:356] "Generic (PLEG): container finished" podID="3e0a71f9-3bdf-42b9-ba6e-94d5d109b483" containerID="54c45890f4aa6e5463350ad49024cfeebb814b6486ce2b9a18ca38103a7bfdb6" exitCode=0
Jul 16 05:31:47.765534 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:47.765157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck" event={"ID":"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483","Type":"ContainerDied","Data":"54c45890f4aa6e5463350ad49024cfeebb814b6486ce2b9a18ca38103a7bfdb6"}
Jul 16 05:31:48.912379 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:48.912355    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:48.989479 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:48.989449    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e0a71f9-3bdf-42b9-ba6e-94d5d109b483-maas-api-service-ca\") pod \"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483\" (UID: \"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483\") "
Jul 16 05:31:48.989901 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:48.989872    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e0a71f9-3bdf-42b9-ba6e-94d5d109b483-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e0a71f9-3bdf-42b9-ba6e-94d5d109b483" (UID: "3e0a71f9-3bdf-42b9-ba6e-94d5d109b483"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:31:49.090387 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:49.090356    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e0a71f9-3bdf-42b9-ba6e-94d5d109b483-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:31:49.777164 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:49.777129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck" event={"ID":"3e0a71f9-3bdf-42b9-ba6e-94d5d109b483","Type":"ContainerDied","Data":"c44a62ad8b01740816e9b04103f5dcca861301070ab899b4fd97bb4487a0f882"}
Jul 16 05:31:49.777164 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:49.777162    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a956af-nrtck"
Jul 16 05:31:49.777374 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:49.777177    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c44a62ad8b01740816e9b04103f5dcca861301070ab899b4fd97bb4487a0f882"
Jul 16 05:31:56.099536 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.099458    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"]
Jul 16 05:31:56.100776 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.100741    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" podUID="61ccc2e3-707a-4d31-9d57-616a1da1ea1a" containerName="maas-api" containerID="cri-o://5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1" gracePeriod=30
Jul 16 05:31:56.126451 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.126423    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:31:56.356348 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.356284    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:31:56.457547 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.457518    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-openshift-service-ca\") pod \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") "
Jul 16 05:31:56.457681 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.457554    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls\") pod \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") "
Jul 16 05:31:56.457681 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.457582    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nlrk\" (UniqueName: \"kubernetes.io/projected/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-kube-api-access-2nlrk\") pod \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\" (UID: \"61ccc2e3-707a-4d31-9d57-616a1da1ea1a\") "
Jul 16 05:31:56.457954 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.457925    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61ccc2e3-707a-4d31-9d57-616a1da1ea1a" (UID: "61ccc2e3-707a-4d31-9d57-616a1da1ea1a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:31:56.459726 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.459705    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-kube-api-access-2nlrk" (OuterVolumeSpecName: "kube-api-access-2nlrk") pod "61ccc2e3-707a-4d31-9d57-616a1da1ea1a" (UID: "61ccc2e3-707a-4d31-9d57-616a1da1ea1a"). InnerVolumeSpecName "kube-api-access-2nlrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:31:56.459784 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.459741    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61ccc2e3-707a-4d31-9d57-616a1da1ea1a" (UID: "61ccc2e3-707a-4d31-9d57-616a1da1ea1a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:31:56.558341 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.558315    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:31:56.558341 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.558340    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:31:56.558488 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.558349    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nlrk\" (UniqueName: \"kubernetes.io/projected/61ccc2e3-707a-4d31-9d57-616a1da1ea1a-kube-api-access-2nlrk\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:31:56.817584 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.817558    2565 generic.go:356] "Generic (PLEG): container finished" podID="61ccc2e3-707a-4d31-9d57-616a1da1ea1a" containerID="5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1" exitCode=0
Jul 16 05:31:56.817709 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.817608    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"
Jul 16 05:31:56.817709 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.817645    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" event={"ID":"61ccc2e3-707a-4d31-9d57-616a1da1ea1a","Type":"ContainerDied","Data":"5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1"}
Jul 16 05:31:56.817709 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.817703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n" event={"ID":"61ccc2e3-707a-4d31-9d57-616a1da1ea1a","Type":"ContainerDied","Data":"e8a51c5a19693e8a390e0f7e2947400591b483c3aa05affcb3684fa1d60f9ef7"}
Jul 16 05:31:56.817818 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.817734    2565 scope.go:117] "RemoveContainer" containerID="5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1"
Jul 16 05:31:56.827159 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.827141    2565 scope.go:117] "RemoveContainer" containerID="5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1"
Jul 16 05:31:56.827442 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:31:56.827423    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1\": container with ID starting with 5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1 not found: ID does not exist" containerID="5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1"
Jul 16 05:31:56.827523 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.827453    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1"} err="failed to get container status \"5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1\": rpc error: code = NotFound desc = could not find container \"5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1\": container with ID starting with 5a922e2003bea3e5f7e3c7bb498687397887b5329dd6d4e0e3a5821d9e19c3c1 not found: ID does not exist"
Jul 16 05:31:56.837907 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.837885    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"]
Jul 16 05:31:56.841509 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:56.841490    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a956af-55cf585f6b-q775n"]
Jul 16 05:31:57.651365 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:31:57.651338    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ccc2e3-707a-4d31-9d57-616a1da1ea1a" path="/var/lib/kubelet/pods/61ccc2e3-707a-4d31-9d57-616a1da1ea1a/volumes"
Jul 16 05:32:00.785186 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:00.785130    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:06.339035 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:06.338995    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:07.845058 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:07.845025    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:11.505524 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.505492    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:11.754412 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.754381    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:11.873384 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873352    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"]
Jul 16 05:32:11.873764 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873751    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e0a71f9-3bdf-42b9-ba6e-94d5d109b483" containerName="revoke-keys"
Jul 16 05:32:11.873813 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873765    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e0a71f9-3bdf-42b9-ba6e-94d5d109b483" containerName="revoke-keys"
Jul 16 05:32:11.873813 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873780    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ccc2e3-707a-4d31-9d57-616a1da1ea1a" containerName="maas-api"
Jul 16 05:32:11.873813 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873786    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ccc2e3-707a-4d31-9d57-616a1da1ea1a" containerName="maas-api"
Jul 16 05:32:11.873940 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873871    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e0a71f9-3bdf-42b9-ba6e-94d5d109b483" containerName="revoke-keys"
Jul 16 05:32:11.873940 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.873880    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ccc2e3-707a-4d31-9d57-616a1da1ea1a" containerName="maas-api"
Jul 16 05:32:11.877097 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.877081    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:11.879455 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.879432    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:32:11.884255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.884230    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"]
Jul 16 05:32:11.999367 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:11.999333    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d82d2b-86c6-4364-988e-ae7a80f6adda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-309054-s2ft7\" (UID: \"98d82d2b-86c6-4364-988e-ae7a80f6adda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:12.100269 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.100230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d82d2b-86c6-4364-988e-ae7a80f6adda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-309054-s2ft7\" (UID: \"98d82d2b-86c6-4364-988e-ae7a80f6adda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:12.100945 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.100919    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d82d2b-86c6-4364-988e-ae7a80f6adda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-309054-s2ft7\" (UID: \"98d82d2b-86c6-4364-988e-ae7a80f6adda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:12.192305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.192232    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:12.331853 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.331808    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"]
Jul 16 05:32:12.333683 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:32:12.333651    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d82d2b_86c6_4364_988e_ae7a80f6adda.slice/crio-0cdf7f240f0ba60f0c37dd2034b956f45e72687ecfe5b39e9a931b887b4ab58f WatchSource:0}: Error finding container 0cdf7f240f0ba60f0c37dd2034b956f45e72687ecfe5b39e9a931b887b4ab58f: Status 404 returned error can't find the container with id 0cdf7f240f0ba60f0c37dd2034b956f45e72687ecfe5b39e9a931b887b4ab58f
Jul 16 05:32:12.372564 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.372540    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:12.474080 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.474012    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:12.906505 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.906470    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7" event={"ID":"98d82d2b-86c6-4364-988e-ae7a80f6adda","Type":"ContainerStarted","Data":"4a839bcca9205762d1c3347dc0ae5f3ace712f279d5182f3ff00b4a9e25e7116"}
Jul 16 05:32:12.906985 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.906528    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7" event={"ID":"98d82d2b-86c6-4364-988e-ae7a80f6adda","Type":"ContainerStarted","Data":"0cdf7f240f0ba60f0c37dd2034b956f45e72687ecfe5b39e9a931b887b4ab58f"}
Jul 16 05:32:12.921431 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:12.921385    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7" podStartSLOduration=1.921370016 podStartE2EDuration="1.921370016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:32:12.919410386 +0000 UTC m=+2529.839993181" watchObservedRunningTime="2026-07-16 05:32:12.921370016 +0000 UTC m=+2529.841952827"
Jul 16 05:32:13.913538 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:13.913506    2565 generic.go:356] "Generic (PLEG): container finished" podID="98d82d2b-86c6-4364-988e-ae7a80f6adda" containerID="4a839bcca9205762d1c3347dc0ae5f3ace712f279d5182f3ff00b4a9e25e7116" exitCode=0
Jul 16 05:32:13.913918 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:13.913582    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7" event={"ID":"98d82d2b-86c6-4364-988e-ae7a80f6adda","Type":"ContainerDied","Data":"4a839bcca9205762d1c3347dc0ae5f3ace712f279d5182f3ff00b4a9e25e7116"}
Jul 16 05:32:15.052306 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.052281    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:15.130894 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.130865    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d82d2b-86c6-4364-988e-ae7a80f6adda-maas-api-service-ca\") pod \"98d82d2b-86c6-4364-988e-ae7a80f6adda\" (UID: \"98d82d2b-86c6-4364-988e-ae7a80f6adda\") "
Jul 16 05:32:15.131189 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.131166    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98d82d2b-86c6-4364-988e-ae7a80f6adda-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "98d82d2b-86c6-4364-988e-ae7a80f6adda" (UID: "98d82d2b-86c6-4364-988e-ae7a80f6adda"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:32:15.232374 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.232307    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d82d2b-86c6-4364-988e-ae7a80f6adda-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:32:15.924172 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.924103    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7"
Jul 16 05:32:15.924172 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.924137    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-309054-s2ft7" event={"ID":"98d82d2b-86c6-4364-988e-ae7a80f6adda","Type":"ContainerDied","Data":"0cdf7f240f0ba60f0c37dd2034b956f45e72687ecfe5b39e9a931b887b4ab58f"}
Jul 16 05:32:15.924348 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:15.924185    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cdf7f240f0ba60f0c37dd2034b956f45e72687ecfe5b39e9a931b887b4ab58f"
Jul 16 05:32:17.291177 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:17.291114    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:17.392218 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:17.392190    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:22.566105 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:22.566069    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:22.665969 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:22.665933    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:22.780977 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:22.780943    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"]
Jul 16 05:32:22.781228 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:22.781196    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" podUID="af9d8124-64b2-4aa3-9520-226c3988f375" containerName="maas-api" containerID="cri-o://a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5" gracePeriod=30
Jul 16 05:32:22.813075 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:22.813032    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:23.237132 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.237104    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:32:23.400244 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.400212    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") pod \"af9d8124-64b2-4aa3-9520-226c3988f375\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") "
Jul 16 05:32:23.400427 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.400271    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af9d8124-64b2-4aa3-9520-226c3988f375-openshift-service-ca\") pod \"af9d8124-64b2-4aa3-9520-226c3988f375\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") "
Jul 16 05:32:23.400427 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.400330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26p5c\" (UniqueName: \"kubernetes.io/projected/af9d8124-64b2-4aa3-9520-226c3988f375-kube-api-access-26p5c\") pod \"af9d8124-64b2-4aa3-9520-226c3988f375\" (UID: \"af9d8124-64b2-4aa3-9520-226c3988f375\") "
Jul 16 05:32:23.400681 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.400654    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af9d8124-64b2-4aa3-9520-226c3988f375-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af9d8124-64b2-4aa3-9520-226c3988f375" (UID: "af9d8124-64b2-4aa3-9520-226c3988f375"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:32:23.402416 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.402384    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af9d8124-64b2-4aa3-9520-226c3988f375-kube-api-access-26p5c" (OuterVolumeSpecName: "kube-api-access-26p5c") pod "af9d8124-64b2-4aa3-9520-226c3988f375" (UID: "af9d8124-64b2-4aa3-9520-226c3988f375"). InnerVolumeSpecName "kube-api-access-26p5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:32:23.402531 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.402435    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af9d8124-64b2-4aa3-9520-226c3988f375" (UID: "af9d8124-64b2-4aa3-9520-226c3988f375"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:32:23.500991 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.500961    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af9d8124-64b2-4aa3-9520-226c3988f375-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:32:23.500991 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.500985    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af9d8124-64b2-4aa3-9520-226c3988f375-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:32:23.501151 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.500998    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26p5c\" (UniqueName: \"kubernetes.io/projected/af9d8124-64b2-4aa3-9520-226c3988f375-kube-api-access-26p5c\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:32:23.972851 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.972752    2565 generic.go:356] "Generic (PLEG): container finished" podID="af9d8124-64b2-4aa3-9520-226c3988f375" containerID="a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5" exitCode=0
Jul 16 05:32:23.972851 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.972797    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"
Jul 16 05:32:23.973294 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.972846    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" event={"ID":"af9d8124-64b2-4aa3-9520-226c3988f375","Type":"ContainerDied","Data":"a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5"}
Jul 16 05:32:23.973294 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.972904    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j" event={"ID":"af9d8124-64b2-4aa3-9520-226c3988f375","Type":"ContainerDied","Data":"9f1d59d70e06125fc7ee48d809ab8415c2176258ff9a433b247ad1e41f071a28"}
Jul 16 05:32:23.973294 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.972928    2565 scope.go:117] "RemoveContainer" containerID="a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5"
Jul 16 05:32:23.982162 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.982145    2565 scope.go:117] "RemoveContainer" containerID="a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5"
Jul 16 05:32:23.982388 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:32:23.982372    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5\": container with ID starting with a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5 not found: ID does not exist" containerID="a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5"
Jul 16 05:32:23.982434 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.982395    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5"} err="failed to get container status \"a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5\": rpc error: code = NotFound desc = could not find container \"a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5\": container with ID starting with a411e370a0cc300e0c3a5d638c969a4681fb629bb8caccff2f30ea77e1a24ee5 not found: ID does not exist"
Jul 16 05:32:23.989659 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.989634    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"]
Jul 16 05:32:23.993067 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:23.993047    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-309054-866fcb4f48-p7z5j"]
Jul 16 05:32:25.614273 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:25.614247    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:25.656623 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:25.656592    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af9d8124-64b2-4aa3-9520-226c3988f375" path="/var/lib/kubelet/pods/af9d8124-64b2-4aa3-9520-226c3988f375/volumes"
Jul 16 05:32:26.959940 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:26.959893    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:32.746782 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:32.746751    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:32.846710 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:32.846683    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:37.690053 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:37.690016    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:37.941283 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:37.941202    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:38.302595 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:38.302516    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:38.645465 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:38.645429    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:39.072385 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:39.072354    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:41.799813 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.799771    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:41.877875 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.877815    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:41.878561 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.878524    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"]
Jul 16 05:32:41.879197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.879178    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98d82d2b-86c6-4364-988e-ae7a80f6adda" containerName="revoke-keys"
Jul 16 05:32:41.879305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.879202    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d82d2b-86c6-4364-988e-ae7a80f6adda" containerName="revoke-keys"
Jul 16 05:32:41.879305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.879225    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af9d8124-64b2-4aa3-9520-226c3988f375" containerName="maas-api"
Jul 16 05:32:41.879305 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.879236    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="af9d8124-64b2-4aa3-9520-226c3988f375" containerName="maas-api"
Jul 16 05:32:41.879467 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.879341    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="98d82d2b-86c6-4364-988e-ae7a80f6adda" containerName="revoke-keys"
Jul 16 05:32:41.879467 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.879363    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="af9d8124-64b2-4aa3-9520-226c3988f375" containerName="maas-api"
Jul 16 05:32:41.883060 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.883040    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:41.885386 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.885366    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ddd4fb-serving-cert\""
Jul 16 05:32:41.892048 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.892025    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"]
Jul 16 05:32:41.946612 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.946583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46348acf-ebb2-4cfc-9f73-a4a656fd4555-openshift-service-ca\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:41.946801 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.946635    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5qc6\" (UniqueName: \"kubernetes.io/projected/46348acf-ebb2-4cfc-9f73-a4a656fd4555-kube-api-access-w5qc6\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:41.946801 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.946773    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:41.986757 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:41.986721    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:42.048347 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.048317    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5qc6\" (UniqueName: \"kubernetes.io/projected/46348acf-ebb2-4cfc-9f73-a4a656fd4555-kube-api-access-w5qc6\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.048498 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.048380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.048498 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.048425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46348acf-ebb2-4cfc-9f73-a4a656fd4555-openshift-service-ca\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.048620 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:32:42.048542    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-serving-cert: secret "maas-api-e2e-api-a-ddd4fb-serving-cert" not found
Jul 16 05:32:42.048620 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:32:42.048617    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls podName:46348acf-ebb2-4cfc-9f73-a4a656fd4555 nodeName:}" failed. No retries permitted until 2026-07-16 05:32:42.548596649 +0000 UTC m=+2559.469179428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls") pod "maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" (UID: "46348acf-ebb2-4cfc-9f73-a4a656fd4555") : secret "maas-api-e2e-api-a-ddd4fb-serving-cert" not found
Jul 16 05:32:42.049031 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.049014    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46348acf-ebb2-4cfc-9f73-a4a656fd4555-openshift-service-ca\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.063215 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.063149    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5qc6\" (UniqueName: \"kubernetes.io/projected/46348acf-ebb2-4cfc-9f73-a4a656fd4555-kube-api-access-w5qc6\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.140958 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.140925    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:42.553047 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.553016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.555537 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.555510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls\") pod \"maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.555976 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.555951    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:42.657597 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.657562    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:42.795445 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.795413    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:42.943987 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.943954    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"]
Jul 16 05:32:42.945346 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:32:42.945322    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46348acf_ebb2_4cfc_9f73_a4a656fd4555.slice/crio-708a661a9412f0c3c6d021ca877807adf6cffd7c49cdcb41dc03825db63b5999 WatchSource:0}: Error finding container 708a661a9412f0c3c6d021ca877807adf6cffd7c49cdcb41dc03825db63b5999: Status 404 returned error can't find the container with id 708a661a9412f0c3c6d021ca877807adf6cffd7c49cdcb41dc03825db63b5999
Jul 16 05:32:42.947131 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:42.947113    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 05:32:43.086268 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:43.086226    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" event={"ID":"46348acf-ebb2-4cfc-9f73-a4a656fd4555","Type":"ContainerStarted","Data":"708a661a9412f0c3c6d021ca877807adf6cffd7c49cdcb41dc03825db63b5999"}
Jul 16 05:32:44.093453 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:44.093410    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" event={"ID":"46348acf-ebb2-4cfc-9f73-a4a656fd4555","Type":"ContainerStarted","Data":"a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d"}
Jul 16 05:32:44.093893 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:44.093485    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:44.110993 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:44.110948    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" podStartSLOduration=2.64193642 podStartE2EDuration="3.110933452s" totalImagesPullingTime="468.997032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:32:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:32:44.107627956 +0000 UTC m=+2561.028210750" watchObservedRunningTime="2026-07-16 05:32:44.110933452 +0000 UTC m=+2561.031516250"
Jul 16 05:32:46.269528 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:46.269484    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:46.785032 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:46.785000    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:47.112435 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:47.112403    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:47.243669 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:47.243629    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:49.717986 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.717941    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:49.834025 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.833986    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:49.847912 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.847884    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"]
Jul 16 05:32:49.851804 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.851785    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:49.854649 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.854626    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2797de-serving-cert\""
Jul 16 05:32:49.874222 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.874197    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"]
Jul 16 05:32:49.922940 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.922910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:49.923107 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.923048    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-openshift-service-ca\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:49.923107 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.923094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48cpq\" (UniqueName: \"kubernetes.io/projected/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-kube-api-access-48cpq\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:49.944088 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:49.944049    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:50.023625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.023540    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-openshift-service-ca\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.023625 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.023586    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48cpq\" (UniqueName: \"kubernetes.io/projected/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-kube-api-access-48cpq\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.023843 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.023645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.023843 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:32:50.023818    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-serving-cert: secret "maas-api-e2e-api-b-2797de-serving-cert" not found
Jul 16 05:32:50.023924 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:32:50.023904    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls podName:05bb5bf5-433b-43ba-bd38-b7c6e1fd915d nodeName:}" failed. No retries permitted until 2026-07-16 05:32:50.523883521 +0000 UTC m=+2567.444466305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls") pod "maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" (UID: "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d") : secret "maas-api-e2e-api-b-2797de-serving-cert" not found
Jul 16 05:32:50.024210 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.024193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-openshift-service-ca\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.034196 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.034165    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48cpq\" (UniqueName: \"kubernetes.io/projected/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-kube-api-access-48cpq\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.104951 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.104925    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:32:50.169172 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.169124    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:50.433924 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.433880    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:50.528227 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.528187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.530730 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.530702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls\") pod \"maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.536402 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.536381    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:50.761867 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.761769    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:50.920321 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:50.920292    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"]
Jul 16 05:32:50.922149 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:32:50.922125    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bb5bf5_433b_43ba_bd38_b7c6e1fd915d.slice/crio-a659aa23b85f14da6fc676ebe42a362a98168e844c5d69c0ddc07bb775373382 WatchSource:0}: Error finding container a659aa23b85f14da6fc676ebe42a362a98168e844c5d69c0ddc07bb775373382: Status 404 returned error can't find the container with id a659aa23b85f14da6fc676ebe42a362a98168e844c5d69c0ddc07bb775373382
Jul 16 05:32:51.133935 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:51.133902    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" event={"ID":"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d","Type":"ContainerStarted","Data":"a659aa23b85f14da6fc676ebe42a362a98168e844c5d69c0ddc07bb775373382"}
Jul 16 05:32:52.140864 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:52.140811    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" event={"ID":"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d","Type":"ContainerStarted","Data":"d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469"}
Jul 16 05:32:52.141248 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:52.140890    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:32:52.156662 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:52.156619    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" podStartSLOduration=2.7543162150000002 podStartE2EDuration="3.156605756s" totalImagesPullingTime="402.289541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:32:52.15480836 +0000 UTC m=+2569.075391154" watchObservedRunningTime="2026-07-16 05:32:52.156605756 +0000 UTC m=+2569.077188558"
Jul 16 05:32:54.268365 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:54.268327    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:32:58.151900 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:32:58.151866    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:33:02.641988 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.641947    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"]
Jul 16 05:33:02.646192 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.646157    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:02.649036 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.649000    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:33:02.653025 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.652997    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"]
Jul 16 05:33:02.732103 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.732064    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9bfe76-ef32-4d03-836a-d5b3446b5780-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl\" (UID: \"6f9bfe76-ef32-4d03-836a-d5b3446b5780\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:02.833085 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.833046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9bfe76-ef32-4d03-836a-d5b3446b5780-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl\" (UID: \"6f9bfe76-ef32-4d03-836a-d5b3446b5780\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:02.833895 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.833870    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9bfe76-ef32-4d03-836a-d5b3446b5780-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl\" (UID: \"6f9bfe76-ef32-4d03-836a-d5b3446b5780\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:02.960436 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:02.960358    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:03.123459 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:03.123422    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"]
Jul 16 05:33:03.124884 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:33:03.124853    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9bfe76_ef32_4d03_836a_d5b3446b5780.slice/crio-b1f989f81624679a45ff82b0faf4b3475e38b92dda173a53e7dcfa7db322b1bd WatchSource:0}: Error finding container b1f989f81624679a45ff82b0faf4b3475e38b92dda173a53e7dcfa7db322b1bd: Status 404 returned error can't find the container with id b1f989f81624679a45ff82b0faf4b3475e38b92dda173a53e7dcfa7db322b1bd
Jul 16 05:33:03.181655 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:03.181631    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:03.203218 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:03.203188    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl" event={"ID":"6f9bfe76-ef32-4d03-836a-d5b3446b5780","Type":"ContainerStarted","Data":"b1f989f81624679a45ff82b0faf4b3475e38b92dda173a53e7dcfa7db322b1bd"}
Jul 16 05:33:03.282808 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:03.282737    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:04.212804 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:04.212710    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f9bfe76-ef32-4d03-836a-d5b3446b5780" containerID="ad344a459a08e0fc0dfa7f68ecba115fe9a16bcbdde17d8d68cbdde8dd9fd82d" exitCode=0
Jul 16 05:33:04.212804 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:04.212790    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl" event={"ID":"6f9bfe76-ef32-4d03-836a-d5b3446b5780","Type":"ContainerDied","Data":"ad344a459a08e0fc0dfa7f68ecba115fe9a16bcbdde17d8d68cbdde8dd9fd82d"}
Jul 16 05:33:05.354154 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:05.354130    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:05.460202 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:05.460170    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9bfe76-ef32-4d03-836a-d5b3446b5780-maas-api-service-ca\") pod \"6f9bfe76-ef32-4d03-836a-d5b3446b5780\" (UID: \"6f9bfe76-ef32-4d03-836a-d5b3446b5780\") "
Jul 16 05:33:05.460520 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:05.460497    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9bfe76-ef32-4d03-836a-d5b3446b5780-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f9bfe76-ef32-4d03-836a-d5b3446b5780" (UID: "6f9bfe76-ef32-4d03-836a-d5b3446b5780"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:33:05.561462 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:05.561404    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9bfe76-ef32-4d03-836a-d5b3446b5780-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:06.224754 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:06.224725    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl"
Jul 16 05:33:06.224946 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:06.224722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl" event={"ID":"6f9bfe76-ef32-4d03-836a-d5b3446b5780","Type":"ContainerDied","Data":"b1f989f81624679a45ff82b0faf4b3475e38b92dda173a53e7dcfa7db322b1bd"}
Jul 16 05:33:06.224946 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:06.224867    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1f989f81624679a45ff82b0faf4b3475e38b92dda173a53e7dcfa7db322b1bd"
Jul 16 05:33:11.328846 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:11.328799    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:11.429754 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:11.429723    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:13.137414 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.137378    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:13.238886 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.238858    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:13.366803 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.366765    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"]
Jul 16 05:33:13.367241 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.367198    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" podUID="46348acf-ebb2-4cfc-9f73-a4a656fd4555" containerName="maas-api" containerID="cri-o://a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d" gracePeriod=30
Jul 16 05:33:13.386841 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.386794    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:13.623138 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.623113    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:33:13.734126 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.734098    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46348acf-ebb2-4cfc-9f73-a4a656fd4555-openshift-service-ca\") pod \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") "
Jul 16 05:33:13.734280 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.734139    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls\") pod \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") "
Jul 16 05:33:13.734335 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.734316    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5qc6\" (UniqueName: \"kubernetes.io/projected/46348acf-ebb2-4cfc-9f73-a4a656fd4555-kube-api-access-w5qc6\") pod \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\" (UID: \"46348acf-ebb2-4cfc-9f73-a4a656fd4555\") "
Jul 16 05:33:13.734456 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.734431    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46348acf-ebb2-4cfc-9f73-a4a656fd4555-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46348acf-ebb2-4cfc-9f73-a4a656fd4555" (UID: "46348acf-ebb2-4cfc-9f73-a4a656fd4555"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:33:13.734701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.734621    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46348acf-ebb2-4cfc-9f73-a4a656fd4555-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:13.736276 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.736255    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46348acf-ebb2-4cfc-9f73-a4a656fd4555" (UID: "46348acf-ebb2-4cfc-9f73-a4a656fd4555"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:33:13.736342 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.736326    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46348acf-ebb2-4cfc-9f73-a4a656fd4555-kube-api-access-w5qc6" (OuterVolumeSpecName: "kube-api-access-w5qc6") pod "46348acf-ebb2-4cfc-9f73-a4a656fd4555" (UID: "46348acf-ebb2-4cfc-9f73-a4a656fd4555"). InnerVolumeSpecName "kube-api-access-w5qc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:33:13.835619 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.835591    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46348acf-ebb2-4cfc-9f73-a4a656fd4555-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:13.835619 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:13.835615    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5qc6\" (UniqueName: \"kubernetes.io/projected/46348acf-ebb2-4cfc-9f73-a4a656fd4555-kube-api-access-w5qc6\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:14.267628 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.267588    2565 generic.go:356] "Generic (PLEG): container finished" podID="46348acf-ebb2-4cfc-9f73-a4a656fd4555" containerID="a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d" exitCode=0
Jul 16 05:33:14.268112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.267648    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"
Jul 16 05:33:14.268112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.267677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" event={"ID":"46348acf-ebb2-4cfc-9f73-a4a656fd4555","Type":"ContainerDied","Data":"a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d"}
Jul 16 05:33:14.268112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.267754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf" event={"ID":"46348acf-ebb2-4cfc-9f73-a4a656fd4555","Type":"ContainerDied","Data":"708a661a9412f0c3c6d021ca877807adf6cffd7c49cdcb41dc03825db63b5999"}
Jul 16 05:33:14.268112 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.267776    2565 scope.go:117] "RemoveContainer" containerID="a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d"
Jul 16 05:33:14.277752 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.277732    2565 scope.go:117] "RemoveContainer" containerID="a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d"
Jul 16 05:33:14.278035 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:33:14.278014    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d\": container with ID starting with a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d not found: ID does not exist" containerID="a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d"
Jul 16 05:33:14.278124 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.278043    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d"} err="failed to get container status \"a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d\": rpc error: code = NotFound desc = could not find container \"a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d\": container with ID starting with a3d58e1742efc44291ffa129d05bffd7002ed3d790c033123ab9733ef3b1de4d not found: ID does not exist"
Jul 16 05:33:14.289707 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.289685    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"]
Jul 16 05:33:14.296607 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:14.296585    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ddd4fb-545f759fd9-7bxnf"]
Jul 16 05:33:15.651204 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:15.651171    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46348acf-ebb2-4cfc-9f73-a4a656fd4555" path="/var/lib/kubelet/pods/46348acf-ebb2-4cfc-9f73-a4a656fd4555/volumes"
Jul 16 05:33:17.726331 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:17.726298    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:23.429054 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:23.429019    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:23.529648 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:23.529610    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:28.460489 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.460401    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:28.715052 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.714927    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:28.836628 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.836590    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"]
Jul 16 05:33:28.837230 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.837210    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46348acf-ebb2-4cfc-9f73-a4a656fd4555" containerName="maas-api"
Jul 16 05:33:28.837283 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.837237    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="46348acf-ebb2-4cfc-9f73-a4a656fd4555" containerName="maas-api"
Jul 16 05:33:28.837283 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.837277    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9bfe76-ef32-4d03-836a-d5b3446b5780" containerName="revoke-keys"
Jul 16 05:33:28.837346 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.837288    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9bfe76-ef32-4d03-836a-d5b3446b5780" containerName="revoke-keys"
Jul 16 05:33:28.837389 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.837378    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9bfe76-ef32-4d03-836a-d5b3446b5780" containerName="revoke-keys"
Jul 16 05:33:28.837427 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.837400    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="46348acf-ebb2-4cfc-9f73-a4a656fd4555" containerName="maas-api"
Jul 16 05:33:28.842433 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.842408    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:28.844813 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.844785    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:33:28.845748 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.845723    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"]
Jul 16 05:33:28.874990 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.874963    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2079d54-45f7-4c22-99ef-ef5fa0a8549e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2797de-lf64d\" (UID: \"b2079d54-45f7-4c22-99ef-ef5fa0a8549e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:28.976787 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.976539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2079d54-45f7-4c22-99ef-ef5fa0a8549e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2797de-lf64d\" (UID: \"b2079d54-45f7-4c22-99ef-ef5fa0a8549e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:28.977260 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:28.977240    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2079d54-45f7-4c22-99ef-ef5fa0a8549e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2797de-lf64d\" (UID: \"b2079d54-45f7-4c22-99ef-ef5fa0a8549e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:29.154442 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:29.154404    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:29.292059 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:29.292035    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"]
Jul 16 05:33:29.294599 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:33:29.294565    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2079d54_45f7_4c22_99ef_ef5fa0a8549e.slice/crio-8f20ea60c47edb56abb423addb5b04d0408bc9e04a906d7bcf29a315c5732b82 WatchSource:0}: Error finding container 8f20ea60c47edb56abb423addb5b04d0408bc9e04a906d7bcf29a315c5732b82: Status 404 returned error can't find the container with id 8f20ea60c47edb56abb423addb5b04d0408bc9e04a906d7bcf29a315c5732b82
Jul 16 05:33:29.349440 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:29.349416    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d" event={"ID":"b2079d54-45f7-4c22-99ef-ef5fa0a8549e","Type":"ContainerStarted","Data":"8f20ea60c47edb56abb423addb5b04d0408bc9e04a906d7bcf29a315c5732b82"}
Jul 16 05:33:29.379351 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:29.379314    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:29.480388 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:29.480359    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:30.356919 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:30.356811    2565 generic.go:356] "Generic (PLEG): container finished" podID="b2079d54-45f7-4c22-99ef-ef5fa0a8549e" containerID="8d01d2c434d235bd01bce691a11f75879748283315f03ea85f83b036813bf508" exitCode=0
Jul 16 05:33:30.356919 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:30.356897    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d" event={"ID":"b2079d54-45f7-4c22-99ef-ef5fa0a8549e","Type":"ContainerDied","Data":"8d01d2c434d235bd01bce691a11f75879748283315f03ea85f83b036813bf508"}
Jul 16 05:33:31.502873 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:31.502790    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:31.604160 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:31.604133    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2079d54-45f7-4c22-99ef-ef5fa0a8549e-maas-api-service-ca\") pod \"b2079d54-45f7-4c22-99ef-ef5fa0a8549e\" (UID: \"b2079d54-45f7-4c22-99ef-ef5fa0a8549e\") "
Jul 16 05:33:31.604499 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:31.604475    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2079d54-45f7-4c22-99ef-ef5fa0a8549e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b2079d54-45f7-4c22-99ef-ef5fa0a8549e" (UID: "b2079d54-45f7-4c22-99ef-ef5fa0a8549e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:33:31.705263 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:31.705233    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2079d54-45f7-4c22-99ef-ef5fa0a8549e-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:32.368713 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:32.368684    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d"
Jul 16 05:33:32.368713 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:32.368699    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2797de-lf64d" event={"ID":"b2079d54-45f7-4c22-99ef-ef5fa0a8549e","Type":"ContainerDied","Data":"8f20ea60c47edb56abb423addb5b04d0408bc9e04a906d7bcf29a315c5732b82"}
Jul 16 05:33:32.369004 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:32.368747    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f20ea60c47edb56abb423addb5b04d0408bc9e04a906d7bcf29a315c5732b82"
Jul 16 05:33:39.333986 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.333950    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"]
Jul 16 05:33:39.334844 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.334789    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" podUID="05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" containerName="maas-api" containerID="cri-o://d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469" gracePeriod=30
Jul 16 05:33:39.361343 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.361300    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:39.597032 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.596978    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:33:39.672710 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.672687    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls\") pod \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") "
Jul 16 05:33:39.672873 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.672746    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48cpq\" (UniqueName: \"kubernetes.io/projected/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-kube-api-access-48cpq\") pod \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") "
Jul 16 05:33:39.672873 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.672865    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-openshift-service-ca\") pod \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\" (UID: \"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d\") "
Jul 16 05:33:39.673206 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.673178    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" (UID: "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:33:39.674631 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.674602    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" (UID: "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:33:39.674721 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.674690    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-kube-api-access-48cpq" (OuterVolumeSpecName: "kube-api-access-48cpq") pod "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" (UID: "05bb5bf5-433b-43ba-bd38-b7c6e1fd915d"). InnerVolumeSpecName "kube-api-access-48cpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:33:39.774100 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.774073    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:39.774100 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.774098    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:39.774240 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:39.774107    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48cpq\" (UniqueName: \"kubernetes.io/projected/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d-kube-api-access-48cpq\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:33:40.412606 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.412570    2565 generic.go:356] "Generic (PLEG): container finished" podID="05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" containerID="d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469" exitCode=0
Jul 16 05:33:40.413036 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.412623    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"
Jul 16 05:33:40.413036 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.412656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" event={"ID":"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d","Type":"ContainerDied","Data":"d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469"}
Jul 16 05:33:40.413036 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.412708    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs" event={"ID":"05bb5bf5-433b-43ba-bd38-b7c6e1fd915d","Type":"ContainerDied","Data":"a659aa23b85f14da6fc676ebe42a362a98168e844c5d69c0ddc07bb775373382"}
Jul 16 05:33:40.413036 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.412732    2565 scope.go:117] "RemoveContainer" containerID="d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469"
Jul 16 05:33:40.422368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.422350    2565 scope.go:117] "RemoveContainer" containerID="d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469"
Jul 16 05:33:40.422606 ip-10-0-138-249 kubenswrapper[2565]: E0716 05:33:40.422587    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469\": container with ID starting with d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469 not found: ID does not exist" containerID="d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469"
Jul 16 05:33:40.422643 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.422614    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469"} err="failed to get container status \"d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469\": rpc error: code = NotFound desc = could not find container \"d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469\": container with ID starting with d56e8b1dac9354cb69bb230a4fc2222b8451fa21b28bac251342696c746ec469 not found: ID does not exist"
Jul 16 05:33:40.432808 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.432789    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"]
Jul 16 05:33:40.437547 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:40.437527    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2797de-7dd8887f94-xjbzs"]
Jul 16 05:33:41.650978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:41.650946    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" path="/var/lib/kubelet/pods/05bb5bf5-433b-43ba-bd38-b7c6e1fd915d/volumes"
Jul 16 05:33:43.919140 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:43.919106    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:49.474197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:49.474161    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:49.574255 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:49.574216    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:54.760511 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:54.760475    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:55.040086 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:55.039995    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:55.454007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:55.453971    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:55.801226 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:55.801148    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:56.482898 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:56.482820    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:59.175790 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.175748    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:59.255766 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.255725    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"]
Jul 16 05:33:59.256296 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.256274    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2079d54-45f7-4c22-99ef-ef5fa0a8549e" containerName="revoke-keys"
Jul 16 05:33:59.256296 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.256297    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2079d54-45f7-4c22-99ef-ef5fa0a8549e" containerName="revoke-keys"
Jul 16 05:33:59.256474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.256321    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" containerName="maas-api"
Jul 16 05:33:59.256474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.256327    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" containerName="maas-api"
Jul 16 05:33:59.256474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.256400    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="05bb5bf5-433b-43ba-bd38-b7c6e1fd915d" containerName="maas-api"
Jul 16 05:33:59.256474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.256407    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2079d54-45f7-4c22-99ef-ef5fa0a8549e" containerName="revoke-keys"
Jul 16 05:33:59.261410 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.261384    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.261537 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.261440    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:59.263701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.263680    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-615edb-serving-cert\""
Jul 16 05:33:59.270550 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.270524    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"]
Jul 16 05:33:59.334604 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.334543    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:59.433771 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.433685    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:33:59.460147 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.460118    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f946e60-490a-4dcc-aea8-0ccd2797063d-maas-api-tls\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.460304 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.460178    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f946e60-490a-4dcc-aea8-0ccd2797063d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.460304 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.460266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxwrs\" (UniqueName: \"kubernetes.io/projected/2f946e60-490a-4dcc-aea8-0ccd2797063d-kube-api-access-zxwrs\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.561143 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.561103    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f946e60-490a-4dcc-aea8-0ccd2797063d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.561343 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.561172    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxwrs\" (UniqueName: \"kubernetes.io/projected/2f946e60-490a-4dcc-aea8-0ccd2797063d-kube-api-access-zxwrs\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.561343 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.561262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f946e60-490a-4dcc-aea8-0ccd2797063d-maas-api-tls\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.561939 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.561907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f946e60-490a-4dcc-aea8-0ccd2797063d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.563764 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.563740    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f946e60-490a-4dcc-aea8-0ccd2797063d-maas-api-tls\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.571745 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.571717    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxwrs\" (UniqueName: \"kubernetes.io/projected/2f946e60-490a-4dcc-aea8-0ccd2797063d-kube-api-access-zxwrs\") pod \"maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.573399 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.573378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:33:59.717013 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.716985    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"]
Jul 16 05:33:59.718735 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:33:59.718708    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f946e60_490a_4dcc_aea8_0ccd2797063d.slice/crio-0b31bfaed6ac51ff207a8a71d83076fd179ba3ded042b4f620723f61c41cc598 WatchSource:0}: Error finding container 0b31bfaed6ac51ff207a8a71d83076fd179ba3ded042b4f620723f61c41cc598: Status 404 returned error can't find the container with id 0b31bfaed6ac51ff207a8a71d83076fd179ba3ded042b4f620723f61c41cc598
Jul 16 05:33:59.922528 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:33:59.922493    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:00.044248 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:00.044127    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:00.525994 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:00.525951    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p" event={"ID":"2f946e60-490a-4dcc-aea8-0ccd2797063d","Type":"ContainerStarted","Data":"7a37581b0a647f419615567aa1dbdb64b5b3c499f8345b72a0dfd22f90de1173"}
Jul 16 05:34:00.526356 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:00.526014    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p" event={"ID":"2f946e60-490a-4dcc-aea8-0ccd2797063d","Type":"ContainerStarted","Data":"0b31bfaed6ac51ff207a8a71d83076fd179ba3ded042b4f620723f61c41cc598"}
Jul 16 05:34:00.526356 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:00.526036    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:34:00.589313 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:00.589261    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p" podStartSLOduration=1.059651979 podStartE2EDuration="1.5892455s" totalImagesPullingTime="529.593521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:34:00.587224063 +0000 UTC m=+2637.507806859" watchObservedRunningTime="2026-07-16 05:34:00.5892455 +0000 UTC m=+2637.509828295"
Jul 16 05:34:03.681936 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:03.681905    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:04.072125 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:04.072073    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:04.703079 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:04.703042    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:04.824007 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:04.823976    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:06.537122 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:06.537091    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:34:07.129655 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.129618    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:07.200243 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.200207    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:07.204624 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.204598    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"]
Jul 16 05:34:07.209726 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.209704    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.213110 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.213090    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e36375-serving-cert\""
Jul 16 05:34:07.220351 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.220330    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"]
Jul 16 05:34:07.231060 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.231034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2695f48e-2369-48e3-9b15-377c8ccebefb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.231162 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.231063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75lwv\" (UniqueName: \"kubernetes.io/projected/2695f48e-2369-48e3-9b15-377c8ccebefb-kube-api-access-75lwv\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.231162 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.231098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2695f48e-2369-48e3-9b15-377c8ccebefb-maas-api-tls\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.264808 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.264767    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:07.331708 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.331677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2695f48e-2369-48e3-9b15-377c8ccebefb-maas-api-tls\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.331874 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.331768    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2695f48e-2369-48e3-9b15-377c8ccebefb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.331874 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.331787    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75lwv\" (UniqueName: \"kubernetes.io/projected/2695f48e-2369-48e3-9b15-377c8ccebefb-kube-api-access-75lwv\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.332510 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.332487    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2695f48e-2369-48e3-9b15-377c8ccebefb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.334286 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.334268    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2695f48e-2369-48e3-9b15-377c8ccebefb-maas-api-tls\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.339583 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.339562    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75lwv\" (UniqueName: \"kubernetes.io/projected/2695f48e-2369-48e3-9b15-377c8ccebefb-kube-api-access-75lwv\") pod \"maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.364502 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.364477    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:07.522010 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.521924    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:07.666924 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.666896    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"]
Jul 16 05:34:07.668273 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:34:07.668247    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2695f48e_2369_48e3_9b15_377c8ccebefb.slice/crio-e90985d541203f4f7a1db05d4ce1d64d2943122d6703a8889709f244d542c920 WatchSource:0}: Error finding container e90985d541203f4f7a1db05d4ce1d64d2943122d6703a8889709f244d542c920: Status 404 returned error can't find the container with id e90985d541203f4f7a1db05d4ce1d64d2943122d6703a8889709f244d542c920
Jul 16 05:34:07.683569 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.683542    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:07.741978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:07.741947    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:08.573200 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:08.573165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq" event={"ID":"2695f48e-2369-48e3-9b15-377c8ccebefb","Type":"ContainerStarted","Data":"145b35ddaed02be5fecac574afe634e04294936b47960393cc0da7ca087ca6ab"}
Jul 16 05:34:08.573373 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:08.573215    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq" event={"ID":"2695f48e-2369-48e3-9b15-377c8ccebefb","Type":"ContainerStarted","Data":"e90985d541203f4f7a1db05d4ce1d64d2943122d6703a8889709f244d542c920"}
Jul 16 05:34:08.573373 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:08.573250    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:08.591056 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:08.590997    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq" podStartSLOduration=1.117595139 podStartE2EDuration="1.590981388s" totalImagesPullingTime="473.386249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:34:08.588125714 +0000 UTC m=+2645.508708521" watchObservedRunningTime="2026-07-16 05:34:08.590981388 +0000 UTC m=+2645.511564182"
Jul 16 05:34:11.689153 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:11.689119    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:14.587163 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:14.587132    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:34:20.763273 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:20.763236    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:20.867070 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:20.867036    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:21.363880 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:21.363821    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:28.440325 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:28.440284    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:28.541012 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:28.540977    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:34:45.656635 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:34:45.656581    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:35:04.440570 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:35:04.440454    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:35:04.446014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:35:04.445995    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:36:32.664452 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:36:32.664367    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:36:32.753368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:36:32.753309    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:36:50.306080 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:36:50.306042    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:36:51.138373 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:36:51.138339    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:36:53.967084 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:36:53.967042    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:37:04.654792 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:37:04.654754    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:37:15.642991 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:37:15.642954    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:37:17.992157 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:37:17.992116    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:37:18.095805 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:37:18.095755    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:38:23.275365 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:38:23.275328    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:38:24.152656 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:38:24.152625    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:38:28.757525 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:38:28.757490    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:38:32.761148 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:38:32.761092    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:38:34.550615 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:38:34.550562    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:38:51.030521 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:38:51.030477    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:21.575799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:21.575761    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:21.686856 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:21.681646    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:29.130258 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:29.130164    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:29.230325 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:29.230286    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:43.059462 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:43.059417    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:43.933969 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:43.933937    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:39:55.021768 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:39:55.021722    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:03.066978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.066938    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"]
Jul 16 05:40:03.070922 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.070900    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:03.073683 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.073656    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:40:03.077086 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.077061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"]
Jul 16 05:40:03.165677 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.165648    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dde12b0-7660-4da8-b2f2-77df9cb0b4cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-615edb-cljbx\" (UID: \"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:03.266551 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.266521    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dde12b0-7660-4da8-b2f2-77df9cb0b4cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-615edb-cljbx\" (UID: \"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:03.267122 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.267104    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dde12b0-7660-4da8-b2f2-77df9cb0b4cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-615edb-cljbx\" (UID: \"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:03.385267 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.385246    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:03.524213 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.524182    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"]
Jul 16 05:40:03.525667 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:40:03.525636    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dde12b0_7660_4da8_b2f2_77df9cb0b4cc.slice/crio-cb05e4fede706199442b12192bfdf8f44c017ecdbfcd048df83e392b2c492b13 WatchSource:0}: Error finding container cb05e4fede706199442b12192bfdf8f44c017ecdbfcd048df83e392b2c492b13: Status 404 returned error can't find the container with id cb05e4fede706199442b12192bfdf8f44c017ecdbfcd048df83e392b2c492b13
Jul 16 05:40:03.551646 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.551618    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx" event={"ID":"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc","Type":"ContainerStarted","Data":"cb05e4fede706199442b12192bfdf8f44c017ecdbfcd048df83e392b2c492b13"}
Jul 16 05:40:03.652993 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.652915    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:03.754127 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:03.754101    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:04.476635 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:04.476487    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:40:04.490603 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:04.483236    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j28v9_c3067477-22a3-452f-b76a-0c7cc590818e/ovn-acl-logging/0.log"
Jul 16 05:40:04.558641 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:04.558619    2565 generic.go:356] "Generic (PLEG): container finished" podID="1dde12b0-7660-4da8-b2f2-77df9cb0b4cc" containerID="840cc5841e62e31039548bced8e5ab9da8be32dd425f93c2723fb1b8887af1d5" exitCode=0
Jul 16 05:40:04.558773 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:04.558649    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx" event={"ID":"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc","Type":"ContainerDied","Data":"840cc5841e62e31039548bced8e5ab9da8be32dd425f93c2723fb1b8887af1d5"}
Jul 16 05:40:05.697379 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:05.697357    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:05.790113 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:05.790086    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dde12b0-7660-4da8-b2f2-77df9cb0b4cc-maas-api-service-ca\") pod \"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc\" (UID: \"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc\") "
Jul 16 05:40:05.790408 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:05.790387    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dde12b0-7660-4da8-b2f2-77df9cb0b4cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1dde12b0-7660-4da8-b2f2-77df9cb0b4cc" (UID: "1dde12b0-7660-4da8-b2f2-77df9cb0b4cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:40:05.891167 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:05.891141    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dde12b0-7660-4da8-b2f2-77df9cb0b4cc-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:06.570117 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:06.570089    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx"
Jul 16 05:40:06.570300 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:06.570093    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-615edb-cljbx" event={"ID":"1dde12b0-7660-4da8-b2f2-77df9cb0b4cc","Type":"ContainerDied","Data":"cb05e4fede706199442b12192bfdf8f44c017ecdbfcd048df83e392b2c492b13"}
Jul 16 05:40:06.570300 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:06.570200    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb05e4fede706199442b12192bfdf8f44c017ecdbfcd048df83e392b2c492b13"
Jul 16 05:40:11.750009 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:11.749977    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:13.441357 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.441320    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"]
Jul 16 05:40:13.442081 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.442045    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p" podUID="2f946e60-490a-4dcc-aea8-0ccd2797063d" containerName="maas-api" containerID="cri-o://7a37581b0a647f419615567aa1dbdb64b5b3c499f8345b72a0dfd22f90de1173" gracePeriod=30
Jul 16 05:40:13.463953 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.463923    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:13.612014 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.611984    2565 generic.go:356] "Generic (PLEG): container finished" podID="2f946e60-490a-4dcc-aea8-0ccd2797063d" containerID="7a37581b0a647f419615567aa1dbdb64b5b3c499f8345b72a0dfd22f90de1173" exitCode=0
Jul 16 05:40:13.612165 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.612053    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p" event={"ID":"2f946e60-490a-4dcc-aea8-0ccd2797063d","Type":"ContainerDied","Data":"7a37581b0a647f419615567aa1dbdb64b5b3c499f8345b72a0dfd22f90de1173"}
Jul 16 05:40:13.704901 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.704845    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:40:13.762106 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.762073    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxwrs\" (UniqueName: \"kubernetes.io/projected/2f946e60-490a-4dcc-aea8-0ccd2797063d-kube-api-access-zxwrs\") pod \"2f946e60-490a-4dcc-aea8-0ccd2797063d\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") "
Jul 16 05:40:13.762266 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.762137    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f946e60-490a-4dcc-aea8-0ccd2797063d-openshift-service-ca\") pod \"2f946e60-490a-4dcc-aea8-0ccd2797063d\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") "
Jul 16 05:40:13.762266 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.762163    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f946e60-490a-4dcc-aea8-0ccd2797063d-maas-api-tls\") pod \"2f946e60-490a-4dcc-aea8-0ccd2797063d\" (UID: \"2f946e60-490a-4dcc-aea8-0ccd2797063d\") "
Jul 16 05:40:13.762444 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.762421    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f946e60-490a-4dcc-aea8-0ccd2797063d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f946e60-490a-4dcc-aea8-0ccd2797063d" (UID: "2f946e60-490a-4dcc-aea8-0ccd2797063d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:40:13.764337 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.764314    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f946e60-490a-4dcc-aea8-0ccd2797063d-kube-api-access-zxwrs" (OuterVolumeSpecName: "kube-api-access-zxwrs") pod "2f946e60-490a-4dcc-aea8-0ccd2797063d" (UID: "2f946e60-490a-4dcc-aea8-0ccd2797063d"). InnerVolumeSpecName "kube-api-access-zxwrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:40:13.764399 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.764368    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f946e60-490a-4dcc-aea8-0ccd2797063d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f946e60-490a-4dcc-aea8-0ccd2797063d" (UID: "2f946e60-490a-4dcc-aea8-0ccd2797063d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:40:13.863798 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.863766    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxwrs\" (UniqueName: \"kubernetes.io/projected/2f946e60-490a-4dcc-aea8-0ccd2797063d-kube-api-access-zxwrs\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:13.863798 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.863792    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f946e60-490a-4dcc-aea8-0ccd2797063d-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:13.863798 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:13.863802    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f946e60-490a-4dcc-aea8-0ccd2797063d-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:14.618522 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:14.618486    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p" event={"ID":"2f946e60-490a-4dcc-aea8-0ccd2797063d","Type":"ContainerDied","Data":"0b31bfaed6ac51ff207a8a71d83076fd179ba3ded042b4f620723f61c41cc598"}
Jul 16 05:40:14.618522 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:14.618513    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"
Jul 16 05:40:14.619032 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:14.618545    2565 scope.go:117] "RemoveContainer" containerID="7a37581b0a647f419615567aa1dbdb64b5b3c499f8345b72a0dfd22f90de1173"
Jul 16 05:40:14.641138 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:14.641112    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"]
Jul 16 05:40:14.644323 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:14.644300    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-615edb-6b65c5655-bdd8p"]
Jul 16 05:40:15.650886 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:15.650855    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f946e60-490a-4dcc-aea8-0ccd2797063d" path="/var/lib/kubelet/pods/2f946e60-490a-4dcc-aea8-0ccd2797063d/volumes"
Jul 16 05:40:18.289667 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:18.289632    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:24.177213 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:24.177172    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:24.240579 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:24.240546    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:28.873235 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:28.873197    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:29.106815 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.106776    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:29.234695 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.234614    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"]
Jul 16 05:40:29.235051 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.235038    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f946e60-490a-4dcc-aea8-0ccd2797063d" containerName="maas-api"
Jul 16 05:40:29.235100 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.235052    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f946e60-490a-4dcc-aea8-0ccd2797063d" containerName="maas-api"
Jul 16 05:40:29.235100 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.235067    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dde12b0-7660-4da8-b2f2-77df9cb0b4cc" containerName="revoke-keys"
Jul 16 05:40:29.235100 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.235072    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dde12b0-7660-4da8-b2f2-77df9cb0b4cc" containerName="revoke-keys"
Jul 16 05:40:29.235197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.235130    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f946e60-490a-4dcc-aea8-0ccd2797063d" containerName="maas-api"
Jul 16 05:40:29.235197 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.235140    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dde12b0-7660-4da8-b2f2-77df9cb0b4cc" containerName="revoke-keys"
Jul 16 05:40:29.239603 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.239585    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:29.242314 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.242130    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqrbr\""
Jul 16 05:40:29.245029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.245005    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"]
Jul 16 05:40:29.317782 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.317749    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/769ed505-525f-4836-a955-09f4d5d9587c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l\" (UID: \"769ed505-525f-4836-a955-09f4d5d9587c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:29.418589 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.418545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/769ed505-525f-4836-a955-09f4d5d9587c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l\" (UID: \"769ed505-525f-4836-a955-09f4d5d9587c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:29.419175 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.419156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/769ed505-525f-4836-a955-09f4d5d9587c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l\" (UID: \"769ed505-525f-4836-a955-09f4d5d9587c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:29.554487 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.554411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:29.695548 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.695515    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"]
Jul 16 05:40:29.697889 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:40:29.697864    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769ed505_525f_4836_a955_09f4d5d9587c.slice/crio-42179b8d23dfa6112c12797fb6563ada12f4dd1d9d604cb29d1ed57a6badc7f1 WatchSource:0}: Error finding container 42179b8d23dfa6112c12797fb6563ada12f4dd1d9d604cb29d1ed57a6badc7f1: Status 404 returned error can't find the container with id 42179b8d23dfa6112c12797fb6563ada12f4dd1d9d604cb29d1ed57a6badc7f1
Jul 16 05:40:29.857234 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.857201    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:29.957922 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:29.957895    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:30.709081 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:30.709042    2565 generic.go:356] "Generic (PLEG): container finished" podID="769ed505-525f-4836-a955-09f4d5d9587c" containerID="71ba3598a0ca53ce87c547802d1fee8f3d06888f339c230931c56809b5a2f1c0" exitCode=0
Jul 16 05:40:30.709218 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:30.709128    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l" event={"ID":"769ed505-525f-4836-a955-09f4d5d9587c","Type":"ContainerDied","Data":"71ba3598a0ca53ce87c547802d1fee8f3d06888f339c230931c56809b5a2f1c0"}
Jul 16 05:40:30.709218 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:30.709177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l" event={"ID":"769ed505-525f-4836-a955-09f4d5d9587c","Type":"ContainerStarted","Data":"42179b8d23dfa6112c12797fb6563ada12f4dd1d9d604cb29d1ed57a6badc7f1"}
Jul 16 05:40:31.857065 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:31.857042    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:31.936584 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:31.936557    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/769ed505-525f-4836-a955-09f4d5d9587c-maas-api-service-ca\") pod \"769ed505-525f-4836-a955-09f4d5d9587c\" (UID: \"769ed505-525f-4836-a955-09f4d5d9587c\") "
Jul 16 05:40:31.936965 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:31.936934    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/769ed505-525f-4836-a955-09f4d5d9587c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "769ed505-525f-4836-a955-09f4d5d9587c" (UID: "769ed505-525f-4836-a955-09f4d5d9587c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:40:31.937074 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:31.937044    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/769ed505-525f-4836-a955-09f4d5d9587c-maas-api-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:32.723820 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:32.723789    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l"
Jul 16 05:40:32.724009 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:32.723818    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l" event={"ID":"769ed505-525f-4836-a955-09f4d5d9587c","Type":"ContainerDied","Data":"42179b8d23dfa6112c12797fb6563ada12f4dd1d9d604cb29d1ed57a6badc7f1"}
Jul 16 05:40:32.724009 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:32.723883    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42179b8d23dfa6112c12797fb6563ada12f4dd1d9d604cb29d1ed57a6badc7f1"
Jul 16 05:40:39.606773 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.606736    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"]
Jul 16 05:40:39.607599 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.607543    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq" podUID="2695f48e-2369-48e3-9b15-377c8ccebefb" containerName="maas-api" containerID="cri-o://145b35ddaed02be5fecac574afe634e04294936b47960393cc0da7ca087ca6ab" gracePeriod=30
Jul 16 05:40:39.629486 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.629443    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:39.765151 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.765116    2565 generic.go:356] "Generic (PLEG): container finished" podID="2695f48e-2369-48e3-9b15-377c8ccebefb" containerID="145b35ddaed02be5fecac574afe634e04294936b47960393cc0da7ca087ca6ab" exitCode=0
Jul 16 05:40:39.765322 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.765192    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq" event={"ID":"2695f48e-2369-48e3-9b15-377c8ccebefb","Type":"ContainerDied","Data":"145b35ddaed02be5fecac574afe634e04294936b47960393cc0da7ca087ca6ab"}
Jul 16 05:40:39.861656 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.861595    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:40:39.895694 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.895670    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75lwv\" (UniqueName: \"kubernetes.io/projected/2695f48e-2369-48e3-9b15-377c8ccebefb-kube-api-access-75lwv\") pod \"2695f48e-2369-48e3-9b15-377c8ccebefb\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") "
Jul 16 05:40:39.895846 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.895702    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2695f48e-2369-48e3-9b15-377c8ccebefb-maas-api-tls\") pod \"2695f48e-2369-48e3-9b15-377c8ccebefb\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") "
Jul 16 05:40:39.895905 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.895883    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2695f48e-2369-48e3-9b15-377c8ccebefb-openshift-service-ca\") pod \"2695f48e-2369-48e3-9b15-377c8ccebefb\" (UID: \"2695f48e-2369-48e3-9b15-377c8ccebefb\") "
Jul 16 05:40:39.896190 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.896166    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2695f48e-2369-48e3-9b15-377c8ccebefb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2695f48e-2369-48e3-9b15-377c8ccebefb" (UID: "2695f48e-2369-48e3-9b15-377c8ccebefb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 05:40:39.897976 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.897951    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2695f48e-2369-48e3-9b15-377c8ccebefb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2695f48e-2369-48e3-9b15-377c8ccebefb" (UID: "2695f48e-2369-48e3-9b15-377c8ccebefb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 05:40:39.897976 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.897958    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2695f48e-2369-48e3-9b15-377c8ccebefb-kube-api-access-75lwv" (OuterVolumeSpecName: "kube-api-access-75lwv") pod "2695f48e-2369-48e3-9b15-377c8ccebefb" (UID: "2695f48e-2369-48e3-9b15-377c8ccebefb"). InnerVolumeSpecName "kube-api-access-75lwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 05:40:39.996811 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.996781    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2695f48e-2369-48e3-9b15-377c8ccebefb-openshift-service-ca\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:39.996933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.996814    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75lwv\" (UniqueName: \"kubernetes.io/projected/2695f48e-2369-48e3-9b15-377c8ccebefb-kube-api-access-75lwv\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:39.996933 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:39.996860    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2695f48e-2369-48e3-9b15-377c8ccebefb-maas-api-tls\") on node \"ip-10-0-138-249.ec2.internal\" DevicePath \"\""
Jul 16 05:40:40.772896 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:40.772865    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"
Jul 16 05:40:40.773347 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:40.772869    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq" event={"ID":"2695f48e-2369-48e3-9b15-377c8ccebefb","Type":"ContainerDied","Data":"e90985d541203f4f7a1db05d4ce1d64d2943122d6703a8889709f244d542c920"}
Jul 16 05:40:40.773347 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:40.772998    2565 scope.go:117] "RemoveContainer" containerID="145b35ddaed02be5fecac574afe634e04294936b47960393cc0da7ca087ca6ab"
Jul 16 05:40:40.795637 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:40.795613    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"]
Jul 16 05:40:40.799585 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:40.799547    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e36375-78c9b44786-c8jxq"]
Jul 16 05:40:41.650799 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:41.650766    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2695f48e-2369-48e3-9b15-377c8ccebefb" path="/var/lib/kubelet/pods/2695f48e-2369-48e3-9b15-377c8ccebefb/volumes"
Jul 16 05:40:44.319125 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:44.319087    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:49.977324 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:49.977293    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:50.079229 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:50.079180    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:55.017161 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:55.017069    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:40:55.243061 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:40:55.243006    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:41:01.386581 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:01.386547    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rh6st_4e348d8d-fc1c-40e5-96d4-9cdf11326572/manager/0.log"
Jul 16 05:41:01.519654 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:01.519622    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5lkgb_821c370a-1f47-4acb-a1ab-64e407151fae/manager/0.log"
Jul 16 05:41:01.656962 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:01.656887    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-679f9cdc6b-b8zfj_caf0b2bd-187f-47b5-a4cf-f7056398dbd7/manager/0.log"
Jul 16 05:41:01.773066 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:01.773041    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gjj6v_2fdf5f77-c3d2-46e4-bc31-42b39a128e30/server/0.log"
Jul 16 05:41:01.916767 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:01.916673    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-996t6_cfa90ba6-1307-4694-8990-a7307b29936c/manager/0.log"
Jul 16 05:41:02.194336 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:02.194249    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-fl2lj_d4f7a532-4aef-4f87-8638-90a4a132d7ac/copy-manifests/0.log"
Jul 16 05:41:02.200693 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:02.200659    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-fl2lj_d4f7a532-4aef-4f87-8638-90a4a132d7ac/manager/0.log"
Jul 16 05:41:02.710129 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:02.710087    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7bd6fbb858-fs7hb_04a29112-991f-46c1-9332-a7723814579e/maas-api/0.log"
Jul 16 05:41:02.839714 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:02.839685    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736315-wcnr2_62813987-835e-4b47-9af2-36b857ff6c64/cleanup/0.log"
Jul 16 05:41:02.965202 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:02.965122    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736330-8qj8h_1a78444b-6b99-4c63-a978-84c178993d04/cleanup/0.log"
Jul 16 05:41:03.083418 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.083380    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-71d327bc-xptbz_ad18a4f5-af2e-4a07-ab85-52d44eec174b/revoke-keys/0.log"
Jul 16 05:41:03.203697 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.203662    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-f2ca692b-wg89m_29b8e867-2d59-4d0d-80be-6be4731356b3/revoke-keys/0.log"
Jul 16 05:41:03.331638 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.331613    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-ddd4fb-r28sl_6f9bfe76-ef32-4d03-836a-d5b3446b5780/revoke-keys/0.log"
Jul 16 05:41:03.448885 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.448857    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-2797de-lf64d_b2079d54-45f7-4c22-99ef-ef5fa0a8549e/revoke-keys/0.log"
Jul 16 05:41:03.566174 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.566146    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-8142960b-wrq2z_dd42d5ff-5d86-4b4c-b2e3-4adb76381ab8/revoke-keys/0.log"
Jul 16 05:41:03.682368 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.682292    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-a956af-nrtck_3e0a71f9-3bdf-42b9-ba6e-94d5d109b483/revoke-keys/0.log"
Jul 16 05:41:03.809062 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.809030    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-309054-s2ft7_98d82d2b-86c6-4364-988e-ae7a80f6adda/revoke-keys/0.log"
Jul 16 05:41:03.926089 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:03.926061    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-1eae9b50-2dr7z_2e6303a1-6b39-4735-9475-ea30a32446b1/revoke-keys/0.log"
Jul 16 05:41:04.044564 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:04.044538    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-18c409ba-vdx9h_3550f49c-0f0f-408d-814c-1f0e026faa48/revoke-keys/0.log"
Jul 16 05:41:04.164145 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:04.164111    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-44053aaf-wp922_7f365a36-f021-425e-bd07-d93b221a16ad/revoke-keys/0.log"
Jul 16 05:41:04.283532 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:04.283444    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-615edb-cljbx_1dde12b0-7660-4da8-b2f2-77df9cb0b4cc/revoke-keys/0.log"
Jul 16 05:41:04.400388 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:04.400354    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-e36375-wjd5l_769ed505-525f-4836-a955-09f4d5d9587c/revoke-keys/0.log"
Jul 16 05:41:04.523961 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:04.523936    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hj6x7_6357802a-c0dd-4c21-9514-24b4fd4ddcc7/postgres/0.log"
Jul 16 05:41:06.048787 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:06.048742    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5j78p_fd1195bc-6a28-49f4-945c-8c2a070527f7/manager/0.log"
Jul 16 05:41:06.170452 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:06.170422    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jzjzd_f982aded-5030-4aad-b09e-9ecceba4450c/manager/0.log"
Jul 16 05:41:06.290083 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:06.290058    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nk4k9_02dc4740-3233-4887-a86d-11a0539ddc8f/kuadrant-console-plugin/0.log"
Jul 16 05:41:06.422907 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:06.422877    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wx7rh_3c2e9023-2c5c-4152-92c2-3020c6073414/registry-server/0.log"
Jul 16 05:41:06.558274 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:06.558246    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-4dscs_f91ccbe1-378f-43f8-ab0d-3d2e961b2890/manager/0.log"
Jul 16 05:41:06.805701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:06.805636    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2dd9r_f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34/manager/0.log"
Jul 16 05:41:07.421964 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:07.421938    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7fc4f54f-4vwml_ed30b757-2ec4-44e4-87ce-8f0c369b6b06/kube-auth-proxy/0.log"
Jul 16 05:41:07.804474 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:07.804402    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wjxsm_338d255d-0874-4e20-aae2-358094dca4fc/payload-pre-processing/0.log"
Jul 16 05:41:08.086520 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:08.086479    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-686d548b76-wnpsf_08124714-a993-46be-85ea-959d06600a08/router/0.log"
Jul 16 05:41:08.487119 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:08.487040    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lqbmc_6192500c-962e-4954-a4a5-d2020c1e219b/main/0.log"
Jul 16 05:41:08.860419 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:08.860385    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbx69pv_3ebb260d-90f4-4ab5-9435-53ea9f40b4ef/main/0.log"
Jul 16 05:41:15.617103 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:15.617060    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:41:15.671033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:15.670985    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log" limit=1024
Jul 16 05:41:18.635655 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:18.635623    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k44rb_c4dfaf89-1298-478a-95ac-7998743f8f51/global-pull-secret-syncer/0.log"
Jul 16 05:41:18.731719 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:18.731677    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4npp2_7c19753c-e312-4509-b103-38488bfcbaa7/konnectivity-agent/0.log"
Jul 16 05:41:18.883389 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:18.883350    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-249.ec2.internal_3a56bf1f2a763dff711431b6502e7b62/haproxy/0.log"
Jul 16 05:41:24.884274 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:24.884165    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5j78p_fd1195bc-6a28-49f4-945c-8c2a070527f7/manager/0.log"
Jul 16 05:41:24.917350 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:24.917308    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jzjzd_f982aded-5030-4aad-b09e-9ecceba4450c/manager/0.log"
Jul 16 05:41:24.943882 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:24.943856    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nk4k9_02dc4740-3233-4887-a86d-11a0539ddc8f/kuadrant-console-plugin/0.log"
Jul 16 05:41:24.998564 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:24.998513    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-wx7rh_3c2e9023-2c5c-4152-92c2-3020c6073414/registry-server/0.log"
Jul 16 05:41:25.225168 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:25.225136    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-4dscs_f91ccbe1-378f-43f8-ab0d-3d2e961b2890/manager/0.log"
Jul 16 05:41:25.351711 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:25.351680    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2dd9r_f38682a4-db2a-4c8e-b05e-1cf4a0c7dd34/manager/0.log"
Jul 16 05:41:27.386329 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.386279    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6xdj_321fede7-f4da-4ba9-bd18-fb521b7c82e1/node-exporter/0.log"
Jul 16 05:41:27.406260 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.406229    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6xdj_321fede7-f4da-4ba9-bd18-fb521b7c82e1/kube-rbac-proxy/0.log"
Jul 16 05:41:27.428092 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.428071    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6xdj_321fede7-f4da-4ba9-bd18-fb521b7c82e1/init-textfile/0.log"
Jul 16 05:41:27.625313 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.625288    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/prometheus/0.log"
Jul 16 05:41:27.668343 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.668316    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/config-reloader/0.log"
Jul 16 05:41:27.689396 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.689371    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/thanos-sidecar/0.log"
Jul 16 05:41:27.712358 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.712342    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/kube-rbac-proxy-web/0.log"
Jul 16 05:41:27.735673 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.735656    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/kube-rbac-proxy/0.log"
Jul 16 05:41:27.757664 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.757646    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/kube-rbac-proxy-thanos/0.log"
Jul 16 05:41:27.778978 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.778962    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_255c40e1-50f8-4583-ba71-edfd9b3393a8/init-config-reloader/0.log"
Jul 16 05:41:27.809454 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.809426    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-w4qpw_6df33176-c55d-4e22-a193-638d28d77620/prometheus-operator/0.log"
Jul 16 05:41:27.828429 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:27.828407    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-w4qpw_6df33176-c55d-4e22-a193-638d28d77620/kube-rbac-proxy/0.log"
Jul 16 05:41:29.328175 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328146    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"]
Jul 16 05:41:29.328710 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328693    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769ed505-525f-4836-a955-09f4d5d9587c" containerName="revoke-keys"
Jul 16 05:41:29.328762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328712    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="769ed505-525f-4836-a955-09f4d5d9587c" containerName="revoke-keys"
Jul 16 05:41:29.328762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328727    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2695f48e-2369-48e3-9b15-377c8ccebefb" containerName="maas-api"
Jul 16 05:41:29.328762 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328732    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2695f48e-2369-48e3-9b15-377c8ccebefb" containerName="maas-api"
Jul 16 05:41:29.328883 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328791    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2695f48e-2369-48e3-9b15-377c8ccebefb" containerName="maas-api"
Jul 16 05:41:29.328883 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.328800    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="769ed505-525f-4836-a955-09f4d5d9587c" containerName="revoke-keys"
Jul 16 05:41:29.331812 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.331796    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.334332 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.334312    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4w7ht\"/\"kube-root-ca.crt\""
Jul 16 05:41:29.335147 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.335125    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4w7ht\"/\"default-dockercfg-wmljj\""
Jul 16 05:41:29.335246 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.335165    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4w7ht\"/\"openshift-service-ca.crt\""
Jul 16 05:41:29.340614 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.340576    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"]
Jul 16 05:41:29.519400 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.519371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rg8\" (UniqueName: \"kubernetes.io/projected/2abd4347-5a36-4b22-9d5a-feecad50a2fb-kube-api-access-p5rg8\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.519550 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.519404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-lib-modules\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.519550 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.519433    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-podres\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.519550 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.519530    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-proc\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.519704 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.519553    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-sys\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.620857 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-proc\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.620857 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620802    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-sys\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620858    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rg8\" (UniqueName: \"kubernetes.io/projected/2abd4347-5a36-4b22-9d5a-feecad50a2fb-kube-api-access-p5rg8\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620889    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-lib-modules\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-proc\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620912    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-sys\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.620950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-podres\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621029 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.621030    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-lib-modules\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.621251 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.621064    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2abd4347-5a36-4b22-9d5a-feecad50a2fb-podres\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.628184 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.628159    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rg8\" (UniqueName: \"kubernetes.io/projected/2abd4347-5a36-4b22-9d5a-feecad50a2fb-kube-api-access-p5rg8\") pod \"perf-node-gather-daemonset-ztxr4\" (UID: \"2abd4347-5a36-4b22-9d5a-feecad50a2fb\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.642764 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.642742    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:29.777494 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:29.777464    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"]
Jul 16 05:41:29.778611 ip-10-0-138-249 kubenswrapper[2565]: W0716 05:41:29.778582    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2abd4347_5a36_4b22_9d5a_feecad50a2fb.slice/crio-0f9e10aabeffd9dde84d5acbced0a2763ed0e8d79dfe9fca04f41f972f200963 WatchSource:0}: Error finding container 0f9e10aabeffd9dde84d5acbced0a2763ed0e8d79dfe9fca04f41f972f200963: Status 404 returned error can't find the container with id 0f9e10aabeffd9dde84d5acbced0a2763ed0e8d79dfe9fca04f41f972f200963
Jul 16 05:41:30.044876 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:30.044848    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4" event={"ID":"2abd4347-5a36-4b22-9d5a-feecad50a2fb","Type":"ContainerStarted","Data":"d25d283092378f7a4228b6331118f67cf72c304af397de6e3199da6f3f9b1a8d"}
Jul 16 05:41:30.045033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:30.044895    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4" event={"ID":"2abd4347-5a36-4b22-9d5a-feecad50a2fb","Type":"ContainerStarted","Data":"0f9e10aabeffd9dde84d5acbced0a2763ed0e8d79dfe9fca04f41f972f200963"}
Jul 16 05:41:30.045033 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:30.044989    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:30.061331 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:30.061282    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4" podStartSLOduration=1.061267373 podStartE2EDuration="1.061267373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 05:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:41:30.059052027 +0000 UTC m=+3086.979634835" watchObservedRunningTime="2026-07-16 05:41:30.061267373 +0000 UTC m=+3086.981850167"
Jul 16 05:41:30.615573 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:30.615535    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-778d57fdbc-g2hxr_f6cd5411-d87a-45fe-976a-9404a1db72e5/console/0.log"
Jul 16 05:41:31.262842 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:31.262805    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-6c9qm_76da998e-d94d-4cb1-ae16-02ed647c23ac/volume-data-source-validator/0.log"
Jul 16 05:41:32.304397 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:32.304361    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8g5qc_4c73832e-cac6-4270-ae47-57021f21b236/dns/0.log"
Jul 16 05:41:32.327209 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:32.327183    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8g5qc_4c73832e-cac6-4270-ae47-57021f21b236/kube-rbac-proxy/0.log"
Jul 16 05:41:32.450935 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:32.450910    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7ttsm_3a9141ae-81d8-40dc-90b5-04a80209ebd4/dns-node-resolver/0.log"
Jul 16 05:41:33.073689 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:33.073661    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-69cb7cb88c-xvzzp_0d72c50d-2fe4-4b32-825b-d114314e1ce4/registry/0.log"
Jul 16 05:41:33.121701 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:33.121664    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k5s9w_a1ea6769-5131-421b-8c6e-bc860c512c25/node-ca/0.log"
Jul 16 05:41:34.257406 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:34.257379    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7fc4f54f-4vwml_ed30b757-2ec4-44e4-87ce-8f0c369b6b06/kube-auth-proxy/0.log"
Jul 16 05:41:34.379921 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:34.379889    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wjxsm_338d255d-0874-4e20-aae2-358094dca4fc/payload-pre-processing/0.log"
Jul 16 05:41:34.437170 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:34.437140    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-686d548b76-wnpsf_08124714-a993-46be-85ea-959d06600a08/router/0.log"
Jul 16 05:41:35.139277 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:35.139251    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brtcs_12db7a2f-2ccd-4bfb-bb1d-27ed64189eb8/serve-healthcheck-canary/0.log"
Jul 16 05:41:35.689168 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:35.689140    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-rt2bg_24f9712e-b284-4b89-a8e7-3898c3ded11f/insights-operator/0.log"
Jul 16 05:41:35.690110 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:35.690086    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-rt2bg_24f9712e-b284-4b89-a8e7-3898c3ded11f/insights-operator/1.log"
Jul 16 05:41:35.782369 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:35.782339    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8tss8_6364c1b9-cca4-43fd-ba81-a829044ceabb/kube-rbac-proxy/0.log"
Jul 16 05:41:35.802635 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:35.802605    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8tss8_6364c1b9-cca4-43fd-ba81-a829044ceabb/exporter/0.log"
Jul 16 05:41:35.823426 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:35.823407    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8tss8_6364c1b9-cca4-43fd-ba81-a829044ceabb/extractor/0.log"
Jul 16 05:41:36.067502 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:36.067481    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-ztxr4"
Jul 16 05:41:38.457879 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:38.457850    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rh6st_4e348d8d-fc1c-40e5-96d4-9cdf11326572/manager/0.log"
Jul 16 05:41:38.706533 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:38.706501    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5lkgb_821c370a-1f47-4acb-a1ab-64e407151fae/manager/0.log"
Jul 16 05:41:38.882144 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:38.882111    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-679f9cdc6b-b8zfj_caf0b2bd-187f-47b5-a4cf-f7056398dbd7/manager/0.log"
Jul 16 05:41:38.902758 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:38.902727    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gjj6v_2fdf5f77-c3d2-46e4-bc31-42b39a128e30/server/0.log"
Jul 16 05:41:39.011504 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:39.011477    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-996t6_cfa90ba6-1307-4694-8990-a7307b29936c/manager/0.log"
Jul 16 05:41:39.129984 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:39.129953    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-fl2lj_d4f7a532-4aef-4f87-8638-90a4a132d7ac/manager/0.log"
Jul 16 05:41:39.149140 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:39.149065    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-fl2lj_d4f7a532-4aef-4f87-8638-90a4a132d7ac/copy-manifests/0.log"
Jul 16 05:41:41.137882 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:41.137854    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7fc4f54f-4vwml_ed30b757-2ec4-44e4-87ce-8f0c369b6b06/kube-auth-proxy/0.log"
Jul 16 05:41:41.237021 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:41.236978    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wjxsm_338d255d-0874-4e20-aae2-358094dca4fc/payload-pre-processing/0.log"
Jul 16 05:41:41.285536 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:41.285504    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-686d548b76-wnpsf_08124714-a993-46be-85ea-959d06600a08/router/0.log"
Jul 16 05:41:42.555369 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:42.555331    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7fc4f54f-4vwml_ed30b757-2ec4-44e4-87ce-8f0c369b6b06/kube-auth-proxy/0.log"
Jul 16 05:41:42.658256 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:42.658229    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-wjxsm_338d255d-0874-4e20-aae2-358094dca4fc/payload-pre-processing/0.log"
Jul 16 05:41:42.706447 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:42.706406    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-686d548b76-wnpsf_08124714-a993-46be-85ea-959d06600a08/router/0.log"
Jul 16 05:41:45.586141 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:45.586112    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9sfzx_311fdd5e-de62-4222-a1cf-63f0690619d6/migrator/0.log"
Jul 16 05:41:45.605539 ip-10-0-138-249 kubenswrapper[2565]: I0716 05:41:45.605512    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9sfzx_311fdd5e-de62-4222-a1cf-63f0690619d6/graceful-termination/0.log"