Jul 15 19:28:28.744378 ip-10-0-130-66 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 19:28:29.190569 ip-10-0-130-66 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:28:29.190569 ip-10-0-130-66 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 19:28:29.190569 ip-10-0-130-66 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:28:29.190569 ip-10-0-130-66 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 19:28:29.190569 ip-10-0-130-66 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:28:29.193222 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.193132    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 19:28:29.199635 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199619    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:28:29.199635 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199635    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199639    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199642    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199646    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199649    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199653    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199655    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199658    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199662    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199667    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199670    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199672    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199675    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199677    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199680    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199682    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199685    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199688    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199690    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199700    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:28:29.199701 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199702    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199706    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199709    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199712    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199714    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199717    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199719    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199722    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199724    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199727    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199729    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199732    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199734    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199737    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199739    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199742    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199744    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199747    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199750    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:28:29.200180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199752    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199755    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199758    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199762    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199765    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199767    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199770    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199773    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199775    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199777    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199780    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199783    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199785    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199788    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199790    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199793    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199795    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199799    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199801    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199804    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:28:29.200629 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199806    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199809    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199812    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199815    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199819    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199822    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199825    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199828    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199830    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199833    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199836    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199839    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199842    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199844    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199847    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199850    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199852    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199854    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199857    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199860    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:28:29.201156 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199862    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199865    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199867    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199870    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199873    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199875    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199877    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199882    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199884    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199887    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199890    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199892    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199894    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199897    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199900    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199902    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199904    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199907    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.199909    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:28:29.201624 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200644    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200650    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200654    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200657    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200660    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200663    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200666    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200669    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200672    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200675    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200677    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200680    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200683    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200686    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200688    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200691    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200695    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200697    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200700    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200702    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:28:29.202107 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200705    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200708    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200711    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200713    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200716    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200719    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200721    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200723    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200726    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200728    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200731    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200733    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200736    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200738    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200740    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200743    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200745    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200749    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200754    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:28:29.202594 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200756    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200776    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200780    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200784    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200787    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200790    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200793    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200797    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200800    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200803    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200806    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200809    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200812    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200815    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200818    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200820    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200823    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200826    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:28:29.203080 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200829    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200831    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200833    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200836    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200839    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200841    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200843    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200846    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200848    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200851    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200853    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200856    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200858    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200861    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200864    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200867    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200870    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200872    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200875    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200877    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:28:29.203517 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200880    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200883    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200885    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200888    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200891    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200893    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200896    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200899    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200901    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200904    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200906    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200909    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200911    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200914    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200916    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200919    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200921    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200924    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200926    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200929    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200931    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:28:29.204132 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.200934    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201310    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201315    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201318    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201320    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201324    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201327    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201329    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201332    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201334    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201336    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201339    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201342    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201344    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201347    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201350    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201352    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201355    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201358    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201361    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:28:29.204638 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201363    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201366    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201368    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201371    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201374    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201376    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201379    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201381    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201384    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201386    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201389    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201391    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201394    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201396    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201399    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201402    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201404    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201407    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201411    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201413    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:28:29.205133 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201415    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201418    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201420    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201423    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201426    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201428    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201431    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201433    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201437    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201439    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201443    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201447    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201449    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201452    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201455    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201458    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201461    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201463    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201466    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:28:29.205650 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201469    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201473    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201476    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201479    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201481    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201484    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201486    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201489    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201491    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201494    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201497    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201501    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201504    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201507    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201510    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201512    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201514    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201517    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201519    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:28:29.206124 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201524    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201527    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201529    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201532    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201534    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201537    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201539    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201542    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201545    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201547    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201549    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201552    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201554    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201557    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201559    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201562    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201564    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201567    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201569    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:28:29.206572 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201571    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201574    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.201576    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201645    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201656    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201662    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201666    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201672    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201676    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201679    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201683    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201686    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201689    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201694    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201697    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201700    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201703    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201705    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201708    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201711    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201714    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201718    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201721    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201724    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 19:28:29.207085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201727    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201730    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201733    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201737    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201739    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201742    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201745    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201748    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201751    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201754    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201757    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201760    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201763    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201766    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201769    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201773    2564 flags.go:64] FLAG: --enable-server="true"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201776    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201780    2564 flags.go:64] FLAG: --event-burst="100"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201783    2564 flags.go:64] FLAG: --event-qps="50"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201786    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201789    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201793    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201797    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201799    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201802    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 19:28:29.207645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201805    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201807    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201810    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201813    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201816    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201818    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201821    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201824    2564 flags.go:64] FLAG: --feature-gates=""
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201828    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201830    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201833    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201836    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201839    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201842    2564 flags.go:64] FLAG: --help="false"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201845    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201848    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201851    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201853    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201857    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201860    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201862    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201865    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201869    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 19:28:29.208248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201872    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201875    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201878    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201881    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201883    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201887    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201890    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201892    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201895    2564 flags.go:64] FLAG: --lock-file=""
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201898    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201900    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201903    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201908    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201911    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201914    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201916    2564 flags.go:64] FLAG: --logging-format="text"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201919    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201922    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201925    2564 flags.go:64] FLAG: --manifest-url=""
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201928    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201932    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201935    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201938    2564 flags.go:64] FLAG: --max-pods="110"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201941    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201944    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 19:28:29.208820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201947    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201950    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201953    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201955    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201958    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201965    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201969    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201972    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201975    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201978    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201983    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201986    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201989    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201993    2564 flags.go:64] FLAG: --port="10250"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201996    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.201998    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ee9701506987199f"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202001    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202004    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202007    2564 flags.go:64] FLAG: --register-node="true"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202010    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202013    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202016    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202018    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202021    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202024    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 19:28:29.209543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202027    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202030    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202033    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202035    2564 flags.go:64] FLAG: --runonce="false"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202038    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202041    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202044    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202059    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202062    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202065    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202068    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202071    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202074    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202078    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202080    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202083    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202086    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202095    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202098    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202103    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202107    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202110    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202114    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202117    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202120    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 19:28:29.210181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202123    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202126    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202129    2564 flags.go:64] FLAG: --v="2"
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202133    2564 flags.go:64] FLAG: --version="false"
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202136    2564 flags.go:64] FLAG: --vmodule=""
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202139    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.202143    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202252    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202256    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202259    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202261    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202264    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202267    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202269    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202272    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202274    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202277    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202280    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202282    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202285    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202288    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:28:29.210783 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202291    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202293    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202296    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202299    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202301    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202304    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202307    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202310    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202312    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202314    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202317    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202319    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202323    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202326    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202329    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202331    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202333    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202336    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202339    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:28:29.211317 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202341    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202343    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202346    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202348    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202351    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202353    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202355    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202358    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202360    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202363    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202365    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202367    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202370    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202373    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202375    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202377    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202380    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202382    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202385    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202388    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:28:29.211951 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202391    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202393    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202395    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202398    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202401    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202403    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202405    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202408    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202411    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202413    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202415    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202419    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202423    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202426    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202428    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202431    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202434    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202436    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202439    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:28:29.212687 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202441    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202444    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202446    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202448    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202451    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202453    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202456    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202458    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202461    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202463    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202465    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202468    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202471    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202473    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202476    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202478    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202481    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202483    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202485    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:28:29.213193 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202489    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202492    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202495    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202498    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202500    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202503    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202505    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.202508    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.203450    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.210568    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.210586    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.210605    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.210611    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210686    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210692    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:28:29.213664 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210695    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210698    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210701    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210704    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210707    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210710    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210712    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210715    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210718    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210721    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210724    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210727    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210730    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210732    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210735    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210738    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210741    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210744    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210746    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210749    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:28:29.214035 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210751    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210754    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210756    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210759    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210762    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210764    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210768    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210773    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210777    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210779    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210782    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210784    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210787    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210790    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210793    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210795    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210798    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210800    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210803    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:28:29.214583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210807    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210810    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210813    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210816    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210819    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210822    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210825    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210827    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210829    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210832    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210834    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210838    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210840    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210842    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210845    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210848    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210850    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210852    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210855    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210857    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:28:29.215066 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210861    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210864    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210866    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210869    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210871    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210873    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210876    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210879    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210881    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210883    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210886    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210888    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210891    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210894    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210896    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210898    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210901    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210904    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210906    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:28:29.215659 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210909    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210911    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210914    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210917    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210921    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210923    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210926    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210928    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210931    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210933    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210936    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210938    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210941    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210944    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210946    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210949    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210952    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210954    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:28:29.216143 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.210957    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.210962    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211096    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211101    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211104    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211107    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211110    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211112    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211115    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211118    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211120    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211124    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211126    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211129    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211131    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:28:29.216582 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211134    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211136    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211138    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211141    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211143    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211146    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211148    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211150    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211153    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211155    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211158    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211160    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211163    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211165    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211168    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211171    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211173    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211176    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211178    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211181    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:28:29.217091 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211183    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211185    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211188    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211190    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211193    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211195    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211197    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211200    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211203    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211205    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211207    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211210    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211212    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211215    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211217    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211220    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211222    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211224    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211227    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211229    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:28:29.217590 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211231    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211234    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211236    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211239    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211241    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211244    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211248    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211251    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211254    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211257    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211260    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211263    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211266    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211268    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211271    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211273    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211275    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211278    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211280    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:28:29.218092 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211282    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211285    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211288    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211290    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211292    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211295    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211297    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211300    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211302    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211305    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211307    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211310    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211312    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211315    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211319    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211322    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211325    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211328    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211331    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:28:29.218552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211335    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211338    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211340    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211343    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211345    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211348    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211350    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:29.211352    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.211357    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.212466    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.215031    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.216658    2564 server.go:1019] "Starting client certificate rotation"
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.216768    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:28:29.219064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.216821    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:28:29.219660 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.219647    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:28:29.220802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.220786    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:28:29.232675 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.232656    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 15 19:28:29.235181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.235168    2564 log.go:25] "Validated CRI v1 image API"
Jul 15 19:28:29.236120 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.236104    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 19:28:29.241975 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.241952    2564 fs.go:135] Filesystem UUIDs: map[5f8c6a5a-41d5-4c84-a38d-06e50141ab0c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d17f3617-bcb8-4008-878f-1ed1afd185c6:/dev/nvme0n1p3]
Jul 15 19:28:29.242024 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.241975    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 19:28:29.246758 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.246740    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:28:29.248188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.248079    2564 manager.go:216] Machine: {Timestamp:2026-07-15 19:28:29.246659835 +0000 UTC m=+0.374622768 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117832 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2614832b8c99d4865d15863bfed501 SystemUUID:ec261483-2b8c-99d4-865d-15863bfed501 BootID:88ebcf17-caf6-4cc9-ac6f-fe3ea963b87c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:98:6b:10:01:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:98:6b:10:01:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:56:0f:ed:0d:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 19:28:29.248188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.248182    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 19:28:29.248298 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.248262    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 19:28:29.252947 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.252922    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 19:28:29.253121 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.252950    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 19:28:29.253864 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.253855    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 19:28:29.253897 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.253866    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 19:28:29.253897 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.253873    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:28:29.253952 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.253896    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:28:29.255600 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.255590    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 19:28:29.259089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259077    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:28:29.259211 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259201    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 19:28:29.259241 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259233    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 19:28:29.259268 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259252    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 19:28:29.259295 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259271    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 19:28:29.259321 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259300    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 15 19:28:29.259365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259354    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 19:28:29.259809 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259754    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:28:29.259809 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.259776    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:28:29.263970 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.263953    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 19:28:29.266269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266254    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 19:28:29.266341 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266286    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 19:28:29.266459 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266449    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 19:28:29.266495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266465    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 19:28:29.266495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266471    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 19:28:29.266495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266477    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 19:28:29.266495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266485    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 19:28:29.266495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.266493    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 19:28:29.267181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.267171    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 19:28:29.267217 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.267181    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 19:28:29.267217 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.267188    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 19:28:29.267217 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.267194    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 19:28:29.267217 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.267202    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 19:28:29.267217 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.267210    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 19:28:29.268879 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.268866    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 19:28:29.268929 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.268882    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 19:28:29.271013 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.270995    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 19:28:29.271109 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.271028    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 19:28:29.271109 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.271044    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 19:28:29.272090 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.272079    2564 server.go:1268] "Started kubelet"
Jul 15 19:28:29.272288 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.272253    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 19:28:29.272991 ip-10-0-130-66 systemd[1]: Started Kubernetes Kubelet.
Jul 15 19:28:29.273593 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.273040    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 19:28:29.273593 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.273125    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 19:28:29.274514 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.274500    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 19:28:29.274758 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.274742    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 19:28:29.277913 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.277896    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 19:28:29.278297 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.277253    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18c28d40336857ea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-07-15 19:28:29.272037354 +0000 UTC m=+0.400000287,LastTimestamp:2026-07-15 19:28:29.272037354 +0000 UTC m=+0.400000287,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Jul 15 19:28:29.283026 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283012    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 19:28:29.283096 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283016    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 19:28:29.283196 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.283177    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.283838 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283818    2564 factory.go:55] Registering systemd factory
Jul 15 19:28:29.283838 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283839    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 15 19:28:29.284019 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283882    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 19:28:29.284019 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283897    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 19:28:29.284019 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.283882    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 19:28:29.284186 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284084    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 19:28:29.284186 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284093    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 19:28:29.284254 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284240    2564 factory.go:153] Registering CRI-O factory
Jul 15 19:28:29.284283 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284263    2564 factory.go:223] Registration of the crio container factory successfully
Jul 15 19:28:29.284332 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284321    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 19:28:29.284380 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284345    2564 factory.go:103] Registering Raw factory
Jul 15 19:28:29.284380 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.284357    2564 manager.go:1196] Started watching for new ooms in manager
Jul 15 19:28:29.285976 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.285961    2564 manager.go:349] Starting recovery of all containers
Jul 15 19:28:29.290751 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.290723    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wxzx"
Jul 15 19:28:29.292710 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.292680    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 19:28:29.292861 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.292838    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 19:28:29.297481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.297325    2564 manager.go:354] Recovery completed
Jul 15 19:28:29.298609 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.298588    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wxzx"
Jul 15 19:28:29.302426 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.302411    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.305204 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.304568    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.305204 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.304597    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.305204 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.304610    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.305926 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.305910    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 19:28:29.305926 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.305924    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 19:28:29.306035 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.305940    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:28:29.306035 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.305960    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.306655 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.306638    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.306769 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.306677    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.306769 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.306689    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.307243 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.307228    2564 policy_none.go:49] "None policy: Start"
Jul 15 19:28:29.307291 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.307249    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 19:28:29.307291 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.307261    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 19:28:29.308282 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.308272    2564 policy_none.go:47] "Start"
Jul 15 19:28:29.345654 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.345635    2564 manager.go:337] "Starting Device Plugin manager"
Jul 15 19:28:29.345773 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.345666    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 19:28:29.345773 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.345676    2564 server.go:87] "Starting device plugin registration server"
Jul 15 19:28:29.345963 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.345924    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 19:28:29.345963 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.345940    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 19:28:29.346406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.346251    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 19:28:29.346406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.346334    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 19:28:29.346406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.346343    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 19:28:29.346770 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.346746    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 19:28:29.346864 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.346808    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.434846 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.434809    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 19:28:29.436183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.436166    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 19:28:29.436277 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.436204    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 19:28:29.436277 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.436228    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 19:28:29.436277 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.436264    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 19:28:29.439195 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.439177    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:28:29.446253 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.446211    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.448347 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.448331    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.448429 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.448372    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.448429 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.448388    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.448429 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.448417    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.457981 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.457967    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.458062 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.457988    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-66.ec2.internal\": node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.504712 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.504694    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.537278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.537255    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"]
Jul 15 19:28:29.537355 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.537314    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.538125 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.538110    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.538177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.538138    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.538177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.538148    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.539439 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.539427    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.539563 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.539548    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.539612 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.539574    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.540188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.540174    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.540273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.540191    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.540273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.540197    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.540273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.540206    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.540273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.540210    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.540273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.540219    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.541300 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.541286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.541372 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.541310    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:28:29.542365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.542350    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:28:29.542365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.542375    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:28:29.542494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.542386    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:28:29.559573 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.559554    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-66.ec2.internal\" not found" node="ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.563807 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.563791    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-66.ec2.internal\" not found" node="ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.586473 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.586453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/525a582238309a692ba219a13a86b2c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"525a582238309a692ba219a13a86b2c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.586564 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.586480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/525a582238309a692ba219a13a86b2c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"525a582238309a692ba219a13a86b2c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.586564 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.586497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54943181b010dce43fa747b7f7cd23b0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"54943181b010dce43fa747b7f7cd23b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.605761 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.605744    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.686876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.686842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/525a582238309a692ba219a13a86b2c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"525a582238309a692ba219a13a86b2c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.686876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.686883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/525a582238309a692ba219a13a86b2c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"525a582238309a692ba219a13a86b2c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.687108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.686901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54943181b010dce43fa747b7f7cd23b0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"54943181b010dce43fa747b7f7cd23b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.687108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.686952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/525a582238309a692ba219a13a86b2c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"525a582238309a692ba219a13a86b2c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.687108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.686961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/525a582238309a692ba219a13a86b2c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"525a582238309a692ba219a13a86b2c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.687108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.686993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54943181b010dce43fa747b7f7cd23b0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"54943181b010dce43fa747b7f7cd23b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.706005 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.705939    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.806777 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.806752    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:29.861979 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.861949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.867770 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:29.867394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:29.907460 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:29.907435    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.008046 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:30.007972    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.108476 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:30.108447    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.209030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:30.208998    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.216168 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.216146    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 19:28:30.216304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.216287    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 19:28:30.244998 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.244978    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:28:30.303308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.303236    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:23:30 +0000 UTC" deadline="2028-04-17 11:43:11.76103451 +0000 UTC"
Jul 15 19:28:30.303308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.303263    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15400h14m41.457774137s"
Jul 15 19:28:30.309396 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:30.309371    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.388965 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.388939    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:28:30.409844 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:30.409815    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.431838 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:30.431328    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54943181b010dce43fa747b7f7cd23b0.slice/crio-d213c18bac3eceae713822c8e8cfcf7e277c0ccd836cc4a176378a801a719eeb WatchSource:0}: Error finding container d213c18bac3eceae713822c8e8cfcf7e277c0ccd836cc4a176378a801a719eeb: Status 404 returned error can't find the container with id d213c18bac3eceae713822c8e8cfcf7e277c0ccd836cc4a176378a801a719eeb
Jul 15 19:28:30.436473 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.436451    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:28:30.437209 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:30.437120    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525a582238309a692ba219a13a86b2c8.slice/crio-0c0a06693031d1c0aa2770096eea07278a8edec8fc49be05c8273be486c7d023 WatchSource:0}: Error finding container 0c0a06693031d1c0aa2770096eea07278a8edec8fc49be05c8273be486c7d023: Status 404 returned error can't find the container with id 0c0a06693031d1c0aa2770096eea07278a8edec8fc49be05c8273be486c7d023
Jul 15 19:28:30.439870 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.439821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"525a582238309a692ba219a13a86b2c8","Type":"ContainerStarted","Data":"0c0a06693031d1c0aa2770096eea07278a8edec8fc49be05c8273be486c7d023"}
Jul 15 19:28:30.440856 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.440833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" event={"ID":"54943181b010dce43fa747b7f7cd23b0","Type":"ContainerStarted","Data":"d213c18bac3eceae713822c8e8cfcf7e277c0ccd836cc4a176378a801a719eeb"}
Jul 15 19:28:30.510640 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:30.510605    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Jul 15 19:28:30.576749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.576676    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:28:30.584108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.584085    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:30.592926 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.592910    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:28:30.593827 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.593816    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Jul 15 19:28:30.606474 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:30.606451    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:28:32.311233 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.311199    2564 apiserver.go:52] "Watching apiserver"
Jul 15 19:28:32.317506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.317482    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 19:28:32.317882 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.317856    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-9xbbj","openshift-network-diagnostics/network-check-target-p2j96","openshift-network-operator/iptables-alerter-mdd44","kube-system/konnectivity-agent-q9675","openshift-dns/node-resolver-jx4q7","openshift-image-registry/node-ca-k29ss","openshift-multus/multus-additional-cni-plugins-ztbhl","openshift-multus/network-metrics-daemon-9pclt","openshift-ovn-kubernetes/ovnkube-node-tlqrq","kube-system/global-pull-secret-syncer-2z2gr","kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9","openshift-cluster-node-tuning-operator/tuned-8b2x9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"]
Jul 15 19:28:32.319495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.319476    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.321041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.321016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.321497 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.321481    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 19:28:32.321888 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.321859    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q9sqs\""
Jul 15 19:28:32.322081 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322063    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.322161 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322103    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.322161 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322131    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 19:28:32.322288 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322103    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 19:28:32.322654 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.322820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322801    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.322932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.322803    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.323340 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.323297    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4pxk6\""
Jul 15 19:28:32.324534 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.324516    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 19:28:32.324942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.324924    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.325161 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.325146    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.325381 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.325363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-psgg2\""
Jul 15 19:28:32.325586 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.325572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.325912 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.325893    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.326318 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.326301    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.328005 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.327989    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.329074 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329055    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.329440 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329421    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 19:28:32.329529 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329493    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.329646 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329620    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mkvtw\""
Jul 15 19:28:32.329735 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329667    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.329735 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329683    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 19:28:32.329735 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329729    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.329949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329804    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 19:28:32.329949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329733    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.329949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329880    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xxc5f\""
Jul 15 19:28:32.329949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.329893    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-69474\""
Jul 15 19:28:32.330099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330061    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.330200 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330181    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 19:28:32.330391 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:32.330391 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330376    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 19:28:32.330506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330403    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 19:28:32.330506 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.330443    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:32.330506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330379    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d5vr5\""
Jul 15 19:28:32.330506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.330497    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.331259 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.331239    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 19:28:32.333913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.332805    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 19:28:32.333913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.333489    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-476jv\""
Jul 15 19:28:32.334469 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.334449    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.334555 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.334526    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 19:28:32.334610 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.334529    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:32.334798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.334779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.334871 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.334854    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:32.337623 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.337601    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.339827 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.339803    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 19:28:32.339912 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.339843    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 19:28:32.339912 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.339901    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 19:28:32.340052 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.340006    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z7xb4\""
Jul 15 19:28:32.344931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.344908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-kubelet\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345095 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.344944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-systemd-units\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345095 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.344978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-cni-bin\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345095 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b69z6\" (UniqueName: \"kubernetes.io/projected/06b65950-8800-4fa8-bfdd-47574402da0d-kube-api-access-b69z6\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345095 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.345305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-lib-modules\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-host\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb6288e1-633c-49b6-a8af-f0c2710493fb-iptables-alerter-script\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.345305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjprw\" (UniqueName: \"kubernetes.io/projected/fb6288e1-633c-49b6-a8af-f0c2710493fb-kube-api-access-fjprw\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-var-lib-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-conf-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345367    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-daemon-config\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-sys\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-kubelet\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-os-release\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jbfj\" (UniqueName: \"kubernetes.io/projected/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-kube-api-access-9jbfj\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.345531 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkmz9\" (UniqueName: \"kubernetes.io/projected/dcb8fe74-76fa-416d-9203-992448ca4ddd-kube-api-access-rkmz9\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345544    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-kubernetes\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345591    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysctl-conf\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345633    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d472w\" (UniqueName: \"kubernetes.io/projected/4e405546-e58e-4c94-a49f-c6c18b7a28d6-kube-api-access-d472w\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-cni-netd\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-system-cni-dir\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-ovn\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-netns\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345745    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-cni-multus\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345781    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-var-lib-kubelet\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-hostroot\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-multus-certs\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-etc-kubernetes\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysctl-d\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.345942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-run\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/041c4a72-8613-4184-afff-0d4be96c0f10-etc-tuned\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.345978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb6288e1-633c-49b6-a8af-f0c2710493fb-host-slash\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30e421da-72b9-4111-afc5-cfcbb4df4731-agent-certs\") pod \"konnectivity-agent-q9675\" (UID: \"30e421da-72b9-4111-afc5-cfcbb4df4731\") " pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-env-overrides\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-os-release\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-cnibin\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346144    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-socket-dir-parent\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346174    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-slash\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-run-netns\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcb8fe74-76fa-416d-9203-992448ca4ddd-serviceca\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346300    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346314    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-kubelet-config\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-systemd\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-node-log\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.346713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-ovnkube-script-lib\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-dbus\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-systemd\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-log-socket\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346525    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-system-cni-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cnibin\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346573    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30e421da-72b9-4111-afc5-cfcbb4df4731-konnectivity-ca\") pod \"konnectivity-agent-q9675\" (UID: \"30e421da-72b9-4111-afc5-cfcbb4df4731\") " pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-ovnkube-config\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346643    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/06b65950-8800-4fa8-bfdd-47574402da0d-ovn-node-metrics-cert\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-cni-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-etc-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8a0c5440-1940-4719-adfb-e10cff36d0f6-cni-binary-copy\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346734    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-cni-bin\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-k8s-cni-cncf-io\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.347463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346780    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e405546-e58e-4c94-a49f-c6c18b7a28d6-hosts-file\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346803    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e405546-e58e-4c94-a49f-c6c18b7a28d6-tmp-dir\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.346887    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-modprobe-d\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347073    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysconfig\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/041c4a72-8613-4184-afff-0d4be96c0f10-tmp\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347148    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clkhs\" (UniqueName: \"kubernetes.io/projected/8a0c5440-1940-4719-adfb-e10cff36d0f6-kube-api-access-clkhs\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcb8fe74-76fa-416d-9203-992448ca4ddd-host\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt6q4\" (UniqueName: \"kubernetes.io/projected/88f9a415-fea9-462f-83bf-e457ab604332-kube-api-access-mt6q4\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.348113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.347274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xzw5\" (UniqueName: \"kubernetes.io/projected/041c4a72-8613-4184-afff-0d4be96c0f10-kube-api-access-8xzw5\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.369309 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.369285    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fnbt4"
Jul 15 19:28:32.376526 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.376507    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fnbt4"
Jul 15 19:28:32.386158 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.386134    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:28:32.435677 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.435651    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 19:28:32.448209 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-var-lib-kubelet\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448209 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-hostroot\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-multus-certs\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-etc-kubernetes\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-device-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-var-lib-kubelet\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-sys-fs\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448321    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-hostroot\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysctl-d\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448400    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-run\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-multus-certs\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448431 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/041c4a72-8613-4184-afff-0d4be96c0f10-etc-tuned\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448470    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-etc-kubernetes\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-run\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb6288e1-633c-49b6-a8af-f0c2710493fb-host-slash\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb6288e1-633c-49b6-a8af-f0c2710493fb-host-slash\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30e421da-72b9-4111-afc5-cfcbb4df4731-agent-certs\") pod \"konnectivity-agent-q9675\" (UID: \"30e421da-72b9-4111-afc5-cfcbb4df4731\") " pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-env-overrides\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysctl-d\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-os-release\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448648    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-cnibin\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-socket-dir-parent\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448730    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448801    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-slash\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.448913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-cnibin\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-socket-dir-parent\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-os-release\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.448756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-slash\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-run-netns\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-run-netns\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcb8fe74-76fa-416d-9203-992448ca4ddd-serviceca\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-kubelet-config\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-env-overrides\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-socket-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-systemd\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-node-log\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-ovnkube-script-lib\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-dbus\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.449605 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-systemd\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-node-log\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-log-socket\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-system-cni-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cnibin\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-kubelet-config\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30e421da-72b9-4111-afc5-cfcbb4df4731-konnectivity-ca\") pod \"konnectivity-agent-q9675\" (UID: \"30e421da-72b9-4111-afc5-cfcbb4df4731\") " pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cnibin\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcb8fe74-76fa-416d-9203-992448ca4ddd-serviceca\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-ovnkube-config\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-systemd\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.449943    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.449969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-dbus\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.450006    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:28:32.949987032 +0000 UTC m=+3.027779828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:32.450407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-log-socket\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450141    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-system-cni-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450172    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/06b65950-8800-4fa8-bfdd-47574402da0d-ovn-node-metrics-cert\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-cni-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-systemd\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450248    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-ovnkube-script-lib\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450273    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-registration-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-cni-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450302    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/06b65950-8800-4fa8-bfdd-47574402da0d-ovnkube-config\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-etc-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-etc-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8a0c5440-1940-4719-adfb-e10cff36d0f6-cni-binary-copy\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-cni-bin\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-k8s-cni-cncf-io\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e405546-e58e-4c94-a49f-c6c18b7a28d6-hosts-file\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e405546-e58e-4c94-a49f-c6c18b7a28d6-tmp-dir\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-cni-bin\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.451208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450533    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e405546-e58e-4c94-a49f-c6c18b7a28d6-hosts-file\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30e421da-72b9-4111-afc5-cfcbb4df4731-konnectivity-ca\") pod \"konnectivity-agent-q9675\" (UID: \"30e421da-72b9-4111-afc5-cfcbb4df4731\") " pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-modprobe-d\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450767    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysconfig\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/041c4a72-8613-4184-afff-0d4be96c0f10-tmp\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-k8s-cni-cncf-io\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450792    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-modprobe-d\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e405546-e58e-4c94-a49f-c6c18b7a28d6-tmp-dir\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.450838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clkhs\" (UniqueName: \"kubernetes.io/projected/8a0c5440-1940-4719-adfb-e10cff36d0f6-kube-api-access-clkhs\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451013    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcb8fe74-76fa-416d-9203-992448ca4ddd-host\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451033    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysconfig\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt6q4\" (UniqueName: \"kubernetes.io/projected/88f9a415-fea9-462f-83bf-e457ab604332-kube-api-access-mt6q4\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xzw5\" (UniqueName: \"kubernetes.io/projected/041c4a72-8613-4184-afff-0d4be96c0f10-kube-api-access-8xzw5\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.452681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcb8fe74-76fa-416d-9203-992448ca4ddd-host\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-kubelet\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-kubelet\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451347    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-systemd-units\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451415    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-systemd-units\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451433    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-cni-bin\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b69z6\" (UniqueName: \"kubernetes.io/projected/06b65950-8800-4fa8-bfdd-47574402da0d-kube-api-access-b69z6\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-cni-bin\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnxkp\" (UniqueName: \"kubernetes.io/projected/363a34fb-8d8e-4862-af6b-7af95ca25a74-kube-api-access-fnxkp\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-lib-modules\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-host\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb6288e1-633c-49b6-a8af-f0c2710493fb-iptables-alerter-script\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjprw\" (UniqueName: \"kubernetes.io/projected/fb6288e1-633c-49b6-a8af-f0c2710493fb-kube-api-access-fjprw\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451660    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-var-lib-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-conf-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-daemon-config\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.453276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-sys\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451786    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-lib-modules\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-kubelet\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-os-release\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-var-lib-openvswitch\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jbfj\" (UniqueName: \"kubernetes.io/projected/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-kube-api-access-9jbfj\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkmz9\" (UniqueName: \"kubernetes.io/projected/dcb8fe74-76fa-416d-9203-992448ca4ddd-kube-api-access-rkmz9\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-kubernetes\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysctl-conf\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d472w\" (UniqueName: \"kubernetes.io/projected/4e405546-e58e-4c94-a49f-c6c18b7a28d6-kube-api-access-d472w\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452016    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-cni-netd\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452029    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-system-cni-dir\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.451659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8a0c5440-1940-4719-adfb-e10cff36d0f6-cni-binary-copy\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-conf-dir\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452029    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-kubelet\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.453798 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.452088    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-kubernetes\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.452138    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:32.952122887 +0000 UTC m=+3.029915672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-host-cni-netd\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-ovn\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-netns\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-system-cni-dir\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452202    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-cni-multus\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb6288e1-633c-49b6-a8af-f0c2710493fb-iptables-alerter-script\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-etc-sysctl-conf\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/06b65950-8800-4fa8-bfdd-47574402da0d-run-ovn\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.454544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-var-lib-cni-multus\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452371    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/041c4a72-8613-4184-afff-0d4be96c0f10-etc-tuned\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-os-release\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.455526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a0c5440-1940-4719-adfb-e10cff36d0f6-host-run-netns\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.452380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-host\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.455910    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.455929    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.455948    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.456020    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:32.955997015 +0000 UTC m=+3.033789814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.456298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8a0c5440-1940-4719-adfb-e10cff36d0f6-multus-daemon-config\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.456400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.456331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30e421da-72b9-4111-afc5-cfcbb4df4731-agent-certs\") pod \"konnectivity-agent-q9675\" (UID: \"30e421da-72b9-4111-afc5-cfcbb4df4731\") " pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.456925 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.456497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/041c4a72-8613-4184-afff-0d4be96c0f10-tmp\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.457576 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.456993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/041c4a72-8613-4184-afff-0d4be96c0f10-sys\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.457576 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.457189    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/06b65950-8800-4fa8-bfdd-47574402da0d-ovn-node-metrics-cert\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.459876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.459851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xzw5\" (UniqueName: \"kubernetes.io/projected/041c4a72-8613-4184-afff-0d4be96c0f10-kube-api-access-8xzw5\") pod \"tuned-8b2x9\" (UID: \"041c4a72-8613-4184-afff-0d4be96c0f10\") " pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.460479 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.460457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjprw\" (UniqueName: \"kubernetes.io/projected/fb6288e1-633c-49b6-a8af-f0c2710493fb-kube-api-access-fjprw\") pod \"iptables-alerter-mdd44\" (UID: \"fb6288e1-633c-49b6-a8af-f0c2710493fb\") " pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.460742 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.460645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.461106 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.461044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b69z6\" (UniqueName: \"kubernetes.io/projected/06b65950-8800-4fa8-bfdd-47574402da0d-kube-api-access-b69z6\") pod \"ovnkube-node-tlqrq\" (UID: \"06b65950-8800-4fa8-bfdd-47574402da0d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.461518 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.461495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt6q4\" (UniqueName: \"kubernetes.io/projected/88f9a415-fea9-462f-83bf-e457ab604332-kube-api-access-mt6q4\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.462427 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.462402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jbfj\" (UniqueName: \"kubernetes.io/projected/fcdbed1e-b25a-49f4-8957-e6687fcaf0f5-kube-api-access-9jbfj\") pod \"multus-additional-cni-plugins-ztbhl\" (UID: \"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5\") " pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.464079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.463395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clkhs\" (UniqueName: \"kubernetes.io/projected/8a0c5440-1940-4719-adfb-e10cff36d0f6-kube-api-access-clkhs\") pod \"multus-9xbbj\" (UID: \"8a0c5440-1940-4719-adfb-e10cff36d0f6\") " pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.465348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.465325    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkmz9\" (UniqueName: \"kubernetes.io/projected/dcb8fe74-76fa-416d-9203-992448ca4ddd-kube-api-access-rkmz9\") pod \"node-ca-k29ss\" (UID: \"dcb8fe74-76fa-416d-9203-992448ca4ddd\") " pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.466255 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.466236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d472w\" (UniqueName: \"kubernetes.io/projected/4e405546-e58e-4c94-a49f-c6c18b7a28d6-kube-api-access-d472w\") pod \"node-resolver-jx4q7\" (UID: \"4e405546-e58e-4c94-a49f-c6c18b7a28d6\") " pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.553593 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnxkp\" (UniqueName: \"kubernetes.io/projected/363a34fb-8d8e-4862-af6b-7af95ca25a74-kube-api-access-fnxkp\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553593 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-device-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-sys-fs\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-sys-fs\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-device-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.553802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-socket-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.554079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-registration-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.554079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553906    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-registration-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.554079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.553944    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/363a34fb-8d8e-4862-af6b-7af95ca25a74-socket-dir\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.561835 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.561810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnxkp\" (UniqueName: \"kubernetes.io/projected/363a34fb-8d8e-4862-af6b-7af95ca25a74-kube-api-access-fnxkp\") pod \"aws-ebs-csi-driver-node-cgtl9\" (UID: \"363a34fb-8d8e-4862-af6b-7af95ca25a74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.634813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.634780    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ztbhl"
Jul 15 19:28:32.640199 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.640170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8b2x9"
Jul 15 19:28:32.649805 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.649781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mdd44"
Jul 15 19:28:32.654340 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.654311    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:32.660875 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.660860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jx4q7"
Jul 15 19:28:32.667383 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.667362    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k29ss"
Jul 15 19:28:32.680928 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.680909    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:32.687505 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.687488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9xbbj"
Jul 15 19:28:32.693017 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.693002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9"
Jul 15 19:28:32.856199 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:32.856170    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb8fe74_76fa_416d_9203_992448ca4ddd.slice/crio-8da7a441e2b6ebf7d3c95ab740a1b396b6919c6e7632cadd6fe4a32c10496186 WatchSource:0}: Error finding container 8da7a441e2b6ebf7d3c95ab740a1b396b6919c6e7632cadd6fe4a32c10496186: Status 404 returned error can't find the container with id 8da7a441e2b6ebf7d3c95ab740a1b396b6919c6e7632cadd6fe4a32c10496186
Jul 15 19:28:32.856552 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:32.856522    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041c4a72_8613_4184_afff_0d4be96c0f10.slice/crio-cb3c692fc47548933e2890207d10f8c12aad374e6bfa4c8609574347bb757ea5 WatchSource:0}: Error finding container cb3c692fc47548933e2890207d10f8c12aad374e6bfa4c8609574347bb757ea5: Status 404 returned error can't find the container with id cb3c692fc47548933e2890207d10f8c12aad374e6bfa4c8609574347bb757ea5
Jul 15 19:28:32.857796 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:32.857776    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6288e1_633c_49b6_a8af_f0c2710493fb.slice/crio-584adbaafe8d94c4509d43483c89e94e21d1766c7364be910d0ed48b0eadaf1e WatchSource:0}: Error finding container 584adbaafe8d94c4509d43483c89e94e21d1766c7364be910d0ed48b0eadaf1e: Status 404 returned error can't find the container with id 584adbaafe8d94c4509d43483c89e94e21d1766c7364be910d0ed48b0eadaf1e
Jul 15 19:28:32.860608 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:32.860546    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdbed1e_b25a_49f4_8957_e6687fcaf0f5.slice/crio-6cf4d164139e1b315de1c4f57713f1b0b0410f38fd1c28e1c77eee85a9804a1d WatchSource:0}: Error finding container 6cf4d164139e1b315de1c4f57713f1b0b0410f38fd1c28e1c77eee85a9804a1d: Status 404 returned error can't find the container with id 6cf4d164139e1b315de1c4f57713f1b0b0410f38fd1c28e1c77eee85a9804a1d
Jul 15 19:28:32.862892 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:28:32.862873    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e405546_e58e_4c94_a49f_c6c18b7a28d6.slice/crio-738f3c93170c1dbbe520b67cfc684d0ff609a9c3548b4fb1b6c12fd38cfad324 WatchSource:0}: Error finding container 738f3c93170c1dbbe520b67cfc684d0ff609a9c3548b4fb1b6c12fd38cfad324: Status 404 returned error can't find the container with id 738f3c93170c1dbbe520b67cfc684d0ff609a9c3548b4fb1b6c12fd38cfad324
Jul 15 19:28:32.956089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.956059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:32.956266 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.956111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:32.956266 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956236    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:32.956266 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956249    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:28:32.956266 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:32.956254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:32.956266 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956270    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:28:32.956498 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956283    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:32.956498 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956301    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:33.956284008 +0000 UTC m=+4.034076803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:32.956498 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956327    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:33.95631225 +0000 UTC m=+4.034105044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:32.956498 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956344    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:32.956498 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:32.956379    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:28:33.956370309 +0000 UTC m=+4.034163088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:33.379468 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.379097    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:23:32 +0000 UTC" deadline="2028-01-20 17:21:04.836124522 +0000 UTC"
Jul 15 19:28:33.379468 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.379357    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13293h52m31.456775225s"
Jul 15 19:28:33.487928 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.487117    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:33.487928 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.487266    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:33.487928 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.487692    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:33.487928 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.487781    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:33.500128 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.498338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k29ss" event={"ID":"dcb8fe74-76fa-416d-9203-992448ca4ddd","Type":"ContainerStarted","Data":"8da7a441e2b6ebf7d3c95ab740a1b396b6919c6e7632cadd6fe4a32c10496186"}
Jul 15 19:28:33.502374 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.502301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"e8024ccd9af443f13e5d9e4341a64f113e6440fc1bd5d2110710f013a8fb5856"}
Jul 15 19:28:33.504681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.504623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mdd44" event={"ID":"fb6288e1-633c-49b6-a8af-f0c2710493fb","Type":"ContainerStarted","Data":"584adbaafe8d94c4509d43483c89e94e21d1766c7364be910d0ed48b0eadaf1e"}
Jul 15 19:28:33.505999 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.505933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8b2x9" event={"ID":"041c4a72-8613-4184-afff-0d4be96c0f10","Type":"ContainerStarted","Data":"cb3c692fc47548933e2890207d10f8c12aad374e6bfa4c8609574347bb757ea5"}
Jul 15 19:28:33.511559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.511534    2564 generic.go:356] "Generic (PLEG): container finished" podID="525a582238309a692ba219a13a86b2c8" containerID="80ff037b1df7e05eb2eee2d8a6c9666e59cd1008c8fe5e99c8d2050bbd5f02a5" exitCode=0
Jul 15 19:28:33.511652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.511605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"525a582238309a692ba219a13a86b2c8","Type":"ContainerDied","Data":"80ff037b1df7e05eb2eee2d8a6c9666e59cd1008c8fe5e99c8d2050bbd5f02a5"}
Jul 15 19:28:33.518494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.518471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9xbbj" event={"ID":"8a0c5440-1940-4719-adfb-e10cff36d0f6","Type":"ContainerStarted","Data":"6b4903eaf2524ef3b9a9b91c4148b13dadbba3022e2cd10f9d51456209e10941"}
Jul 15 19:28:33.520677 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.520656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q9675" event={"ID":"30e421da-72b9-4111-afc5-cfcbb4df4731","Type":"ContainerStarted","Data":"ba99636b7df5e918cb512dbcefe609d60f981501242089ada8f2855ba874caff"}
Jul 15 19:28:33.528089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.528060    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9" event={"ID":"363a34fb-8d8e-4862-af6b-7af95ca25a74","Type":"ContainerStarted","Data":"57229efe89cd37331ee5f0b317c19ea25e4cc9f8ca7c6632c1d17e132ee12fc1"}
Jul 15 19:28:33.537171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.537145    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jx4q7" event={"ID":"4e405546-e58e-4c94-a49f-c6c18b7a28d6","Type":"ContainerStarted","Data":"738f3c93170c1dbbe520b67cfc684d0ff609a9c3548b4fb1b6c12fd38cfad324"}
Jul 15 19:28:33.539499 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.539475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerStarted","Data":"6cf4d164139e1b315de1c4f57713f1b0b0410f38fd1c28e1c77eee85a9804a1d"}
Jul 15 19:28:33.965328 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.965234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:33.965328 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.965303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:33.965532 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:33.965334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:33.965532 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.965452    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:33.965532 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.965510    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:28:35.965492737 +0000 UTC m=+6.043285519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:33.965910 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.965889    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:33.965999 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.965945    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:35.965930685 +0000 UTC m=+6.043723477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:33.966056 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.966024    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:28:33.966056 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.966041    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:28:33.966056 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.966053    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:33.966196 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:33.966098    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:35.966085916 +0000 UTC m=+6.043878696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:34.380331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:34.380199    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:23:32 +0000 UTC" deadline="2028-02-29 20:03:27.015709048 +0000 UTC"
Jul 15 19:28:34.380331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:34.380279    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14256h34m52.63543416s"
Jul 15 19:28:34.491187 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:34.490701    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:34.491187 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:34.490835    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:34.546855 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:34.546021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"525a582238309a692ba219a13a86b2c8","Type":"ContainerStarted","Data":"fc453788f82b9945c42b76905814bb0cc86604fde0d2e0300520464109ba07e9"}
Jul 15 19:28:34.561415 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:34.560951    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" podStartSLOduration=3.560932159 podStartE2EDuration="3.560932159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:34.560061117 +0000 UTC m=+4.637853919" watchObservedRunningTime="2026-07-15 19:28:34.560932159 +0000 UTC m=+4.638724962"
Jul 15 19:28:35.487573 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:35.487501    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:35.488036 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.487637    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:35.488036 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:35.487960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:35.488309 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.488044    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:35.982924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:35.982980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:35.983031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983157    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983232    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:39.983197422 +0000 UTC m=+10.060990205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983686    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983706    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983719    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983765    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:39.983748582 +0000 UTC m=+10.061541383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983826    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:35.983938 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:35.983856    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:28:39.983846013 +0000 UTC m=+10.061638801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:36.489702 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:36.488826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:36.489702 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:36.488970    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:37.487829 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:37.487318    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:37.487829 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:37.487349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:37.487829 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:37.487433    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:37.487829 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:37.487785    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:38.487401 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:38.487356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:38.487781 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:38.487506    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:39.486896 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:39.486818    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:39.487052 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:39.486954    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:39.487478 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:39.487344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:39.487478 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:39.487437    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:40.021265 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:40.021209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:40.021456 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:40.021289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:40.021456 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:40.021346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:40.021565 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.021467    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:40.021565 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.021528    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:48.021508455 +0000 UTC m=+18.099301238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:40.021962 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.021943    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:28:40.022017 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.021972    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:28:40.022017 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.021985    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:40.022115 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.022031    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:48.022015811 +0000 UTC m=+18.099808603 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:40.022115 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.022093    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:40.022199 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.022127    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:28:48.02211708 +0000 UTC m=+18.099909865 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:40.489325 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:40.489247    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:40.489779 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:40.489376    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:41.487250 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:41.487197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:41.487413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:41.487255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:41.487413 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:41.487345    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:41.487514 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:41.487478    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:42.487064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:42.487029    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:42.487553 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:42.487173    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:43.487336 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:43.487297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:43.487784 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:43.487311    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:43.487784 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:43.487411    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:43.487784 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:43.487517    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:44.487102 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:44.487070    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:44.487305 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:44.487235    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:45.487044 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:45.487005    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:45.487044 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:45.487046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:45.487513 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:45.487136    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:45.487513 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:45.487306    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:46.487734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:46.487698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:46.488205 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:46.487847    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:47.487418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:47.487380    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:47.487418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:47.487412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:47.487662 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:47.487491    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:47.487833 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:47.487803    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:48.080503 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:48.080468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:48.080667 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:48.080513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:48.080667 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:48.080536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:48.080667 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080611    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:48.080667 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080634    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:48.080667 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080650    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:28:48.080667 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080667    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:28:48.080881 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080676    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:04.080661494 +0000 UTC m=+34.158454273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:28:48.080881 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080678    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:48.080881 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080691    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:04.080685196 +0000 UTC m=+34.158477975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:28:48.080881 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.080719    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:04.080706345 +0000 UTC m=+34.158499124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:28:48.487493 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:48.487415    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:48.487634 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:48.487560    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:49.487467 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:49.487433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:49.487862 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:49.487433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:49.487862 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:49.487546    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:49.487862 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:49.487638    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:50.487979 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:50.487944    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:50.488363 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:50.488051    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:51.487548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:51.487520    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:51.487790 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:51.487574    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:51.487790 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:51.487676    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:51.487909 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:51.487816    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:52.493007 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.492154    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:52.493007 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:52.492497    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:52.586021 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.585991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9" event={"ID":"363a34fb-8d8e-4862-af6b-7af95ca25a74","Type":"ContainerStarted","Data":"d47e5738293755055e5bec72c43db0294b8634adac6b3f07fcafecd0ce9c210c"}
Jul 15 19:28:52.588042 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.588001    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jx4q7" event={"ID":"4e405546-e58e-4c94-a49f-c6c18b7a28d6","Type":"ContainerStarted","Data":"02cab93ac9069d55fad3e6c83bc517dd758a20d49231667253909f76dc19e43a"}
Jul 15 19:28:52.589607 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.589444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerStarted","Data":"c2ac27195c514120dadcb5c5aea1c4d07e105d5826a46da04a0369200deeed36"}
Jul 15 19:28:52.592570 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.592545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k29ss" event={"ID":"dcb8fe74-76fa-416d-9203-992448ca4ddd","Type":"ContainerStarted","Data":"20b2a3ef3af2c658d83d7f9d29960e591a73b24e943876285bb588d0cb66fba3"}
Jul 15 19:28:52.596474 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.596444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"1a18bf14745a19a24e7a3681699b5bc76ecccf752b89747a56b9bb2cfebd0f18"}
Jul 15 19:28:52.598815 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.598786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8b2x9" event={"ID":"041c4a72-8613-4184-afff-0d4be96c0f10","Type":"ContainerStarted","Data":"1078a022c2660c6346b1092a70417996de43160830e591182eba59d6ce896aad"}
Jul 15 19:28:52.601054 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.601004    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jx4q7" podStartSLOduration=3.142168932 podStartE2EDuration="22.600988174s" totalImagesPullingTime="19.458819242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:52.600741908 +0000 UTC m=+22.678534711" watchObservedRunningTime="2026-07-15 19:28:52.600988174 +0000 UTC m=+22.678780970"
Jul 15 19:28:52.601557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.601488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" event={"ID":"54943181b010dce43fa747b7f7cd23b0","Type":"ContainerStarted","Data":"366408fa0380a4aee4e9b3bf067ae2bfde82cb7f15def2163a460f3cdd9bf91d"}
Jul 15 19:28:52.604506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.603958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9xbbj" event={"ID":"8a0c5440-1940-4719-adfb-e10cff36d0f6","Type":"ContainerStarted","Data":"bd9cd04c7780540820c05d4406cdd28ed47348436fec69831d182b2b0a3bb585"}
Jul 15 19:28:52.606183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.606162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q9675" event={"ID":"30e421da-72b9-4111-afc5-cfcbb4df4731","Type":"ContainerStarted","Data":"fa1eb047ee4a9108d5628b2e9b907f11b3b94350da9604ea812b034bf8873837"}
Jul 15 19:28:52.614417 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.614380    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k29ss" podStartSLOduration=3.149854866 podStartE2EDuration="22.614366143s" totalImagesPullingTime="19.464511277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:52.614148829 +0000 UTC m=+22.691941627" watchObservedRunningTime="2026-07-15 19:28:52.614366143 +0000 UTC m=+22.692158983"
Jul 15 19:28:52.650743 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.650690    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8b2x9" podStartSLOduration=3.184003709 podStartE2EDuration="22.650674217s" totalImagesPullingTime="19.466670508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:52.650239303 +0000 UTC m=+22.728032103" watchObservedRunningTime="2026-07-15 19:28:52.650674217 +0000 UTC m=+22.728467044"
Jul 15 19:28:52.664749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.664687    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q9675" podStartSLOduration=8.577961357 podStartE2EDuration="22.66467157s" totalImagesPullingTime="14.086710213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:52.664407969 +0000 UTC m=+22.742200771" watchObservedRunningTime="2026-07-15 19:28:52.66467157 +0000 UTC m=+22.742464370"
Jul 15 19:28:52.686158 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.686115    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" podStartSLOduration=21.686100037 podStartE2EDuration="21.686100037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:52.685736552 +0000 UTC m=+22.763529353" watchObservedRunningTime="2026-07-15 19:28:52.686100037 +0000 UTC m=+22.763892838"
Jul 15 19:28:52.730788 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:52.730748    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9xbbj" podStartSLOduration=3.278885119 podStartE2EDuration="22.73073233s" totalImagesPullingTime="19.451847211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:52.730713303 +0000 UTC m=+22.808506104" watchObservedRunningTime="2026-07-15 19:28:52.73073233 +0000 UTC m=+22.808525125"
Jul 15 19:28:53.487281 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.487243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:53.487472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.487241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:53.487472 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:53.487352    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:53.487472 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:53.487419    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:53.610238 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.610143    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcdbed1e-b25a-49f4-8957-e6687fcaf0f5" containerID="c2ac27195c514120dadcb5c5aea1c4d07e105d5826a46da04a0369200deeed36" exitCode=0
Jul 15 19:28:53.611028 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.610254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerDied","Data":"c2ac27195c514120dadcb5c5aea1c4d07e105d5826a46da04a0369200deeed36"}
Jul 15 19:28:53.613108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613092    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:28:53.613446 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613428    2564 generic.go:356] "Generic (PLEG): container finished" podID="06b65950-8800-4fa8-bfdd-47574402da0d" containerID="74f3205682a148fe0c8c923fd885cb07b63377512f3115b3c3a4d207e8aa850e" exitCode=1
Jul 15 19:28:53.613554 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613532    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"9b05c7bc5dda9cef12d17bd7bae2c69e585b9ca7ecd764ed0a1837e46421c61a"}
Jul 15 19:28:53.613620 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"dbb1440c3333a917eb2bf2c543db2f4a8dfea1745ebb6b93b7390eeb06845716"}
Jul 15 19:28:53.613664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613647    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"2070641caac8e67c86509858dd65cbe646ae84d4857618b67163199888c9c429"}
Jul 15 19:28:53.613711 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"0f1dab0026c2da93cf7ea405e99ddbb299a1680ecc7fffff39c5ff9670a6977e"}
Jul 15 19:28:53.613764 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.613722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerDied","Data":"74f3205682a148fe0c8c923fd885cb07b63377512f3115b3c3a4d207e8aa850e"}
Jul 15 19:28:53.896493 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:53.896466    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 19:28:54.407767 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.407644    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:28:53.896485029Z","UUID":"d82ee9da-62ce-4364-8446-86ad5f71f9f5","Handler":null,"Name":"","Endpoint":""}
Jul 15 19:28:54.409189 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.409169    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 19:28:54.409321 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.409203    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 19:28:54.487361 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.487310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:54.487528 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:54.487452    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:54.617750 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.617714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9" event={"ID":"363a34fb-8d8e-4862-af6b-7af95ca25a74","Type":"ContainerStarted","Data":"d6001f9c299c24412f2a082ca5194fadfd4b94cafeadd5a36b326f2214a61607"}
Jul 15 19:28:54.619429 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.619397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mdd44" event={"ID":"fb6288e1-633c-49b6-a8af-f0c2710493fb","Type":"ContainerStarted","Data":"d9f3f6d16ceb744b31d8ee40526b4f1bd8b4382072b760186c9b0998c967b97f"}
Jul 15 19:28:54.632626 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:54.632578    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mdd44" podStartSLOduration=10.536208586 podStartE2EDuration="24.632558725s" totalImagesPullingTime="14.096350139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:54.631963148 +0000 UTC m=+24.709755948" watchObservedRunningTime="2026-07-15 19:28:54.632558725 +0000 UTC m=+24.710351525"
Jul 15 19:28:55.427354 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:55.427058    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:55.487571 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:55.487535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:55.487754 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:55.487548    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:55.487754 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:55.487662    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:55.487754 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:55.487739    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:55.625599 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:55.625568    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:28:55.626168 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:55.626049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"8ccf0ffb575a2ba2e669e854af0703746e281e2005e9d1a9000f087920a34d75"}
Jul 15 19:28:56.489884 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:56.489852    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:56.490024 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:56.489991    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:56.631338 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:56.631301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9" event={"ID":"363a34fb-8d8e-4862-af6b-7af95ca25a74","Type":"ContainerStarted","Data":"d9944eeced0c4e92c8dc92c965abd2b69c06ecafab352569f72a45afaef37f60"}
Jul 15 19:28:56.646637 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:56.646597    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgtl9" podStartSLOduration=3.87526487 podStartE2EDuration="26.646580521s" totalImagesPullingTime="22.771315651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:56.645888364 +0000 UTC m=+26.723681166" watchObservedRunningTime="2026-07-15 19:28:56.646580521 +0000 UTC m=+26.724373316"
Jul 15 19:28:57.454777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.454696    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:57.455324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.455300    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:57.487533 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.487506    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:57.487702 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:57.487632    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:57.487702 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.487687    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:57.487805 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:57.487792    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:57.643894 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.643345    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:28:57.645099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.643926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"dad2b322c3411de1337f7b88045bce1a9d312bc19ee9275083aa08146d2470fe"}
Jul 15 19:28:57.645099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.644486    2564 scope.go:117] "RemoveContainer" containerID="74f3205682a148fe0c8c923fd885cb07b63377512f3115b3c3a4d207e8aa850e"
Jul 15 19:28:57.645556 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:57.645120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q9675"
Jul 15 19:28:58.487199 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.487022    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:58.487395 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:58.487309    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:58.648146 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.648115    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcdbed1e-b25a-49f4-8957-e6687fcaf0f5" containerID="7036fc0ba90509c44ef0d3b964ba15aed0e95537c312db83ade98cb5ce38d3e2" exitCode=0
Jul 15 19:28:58.648579 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.648205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerDied","Data":"7036fc0ba90509c44ef0d3b964ba15aed0e95537c312db83ade98cb5ce38d3e2"}
Jul 15 19:28:58.651928 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.651910    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:28:58.652292 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.652255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" event={"ID":"06b65950-8800-4fa8-bfdd-47574402da0d","Type":"ContainerStarted","Data":"e58c9a2e9f377cd9a86e863cb8c06802f93e43edc19c0b76d260c4f0874eac51"}
Jul 15 19:28:58.652486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.652469    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:58.652561 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.652548    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:58.652614 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.652602    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:58.668172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.668151    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:58.668488 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.668452    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:28:58.692861 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:58.692813    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq" podStartSLOduration=9.138681208 podStartE2EDuration="28.692799569s" totalImagesPullingTime="19.554118361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:58.692450261 +0000 UTC m=+28.770243062" watchObservedRunningTime="2026-07-15 19:28:58.692799569 +0000 UTC m=+28.770592390"
Jul 15 19:28:59.486741 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.486712    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:59.486741 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.486751    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:59.486929 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:59.486829    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:28:59.486974 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:59.486948    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:59.553502 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.553442    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p2j96"]
Jul 15 19:28:59.557005 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.556976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9pclt"]
Jul 15 19:28:59.557129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.557079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:28:59.557202 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:59.557181    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:28:59.557759 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.557737    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z2gr"]
Jul 15 19:28:59.654915 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.654881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:28:59.655385 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:28:59.654881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:28:59.655385 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:59.655008    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:28:59.655385 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:28:59.655075    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:29:00.658945 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:00.658757    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcdbed1e-b25a-49f4-8957-e6687fcaf0f5" containerID="b459ae03135323a184ffaaef6717184afabd266420763f15ec1f8eab5b30c02f" exitCode=0
Jul 15 19:29:00.659304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:00.658837    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerDied","Data":"b459ae03135323a184ffaaef6717184afabd266420763f15ec1f8eab5b30c02f"}
Jul 15 19:29:01.486909 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:01.486826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:29:01.486909 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:01.486853    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:01.486909 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:01.486855    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:01.487123 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:01.486927    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:29:01.487123 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:01.487017    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:29:01.487123 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:01.487093    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:29:03.486934 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:03.486898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:29:03.487590 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:03.486898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:03.487590 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:03.487014    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9pclt" podUID="88f9a415-fea9-462f-83bf-e457ab604332"
Jul 15 19:29:03.487590 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:03.486899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:03.487590 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:03.487088    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p2j96" podUID="50ceb82a-401a-4b46-854e-de72397e5bc9"
Jul 15 19:29:03.487590 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:03.487146    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z2gr" podUID="dbb785a9-ed6f-4480-bd9c-e190191eaf8e"
Jul 15 19:29:03.667891 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:03.667808    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcdbed1e-b25a-49f4-8957-e6687fcaf0f5" containerID="480218e50761c3afd4ffbbaecb84afcea9b996affe8760a777b40c2ce3e2c5f6" exitCode=0
Jul 15 19:29:03.667891 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:03.667856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerDied","Data":"480218e50761c3afd4ffbbaecb84afcea9b996affe8760a777b40c2ce3e2c5f6"}
Jul 15 19:29:04.099857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:04.099825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:04.099867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:04.099887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.099963    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.099967    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.099998    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.100013    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret podName:dbb785a9-ed6f-4480-bd9c-e190191eaf8e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:36.099999608 +0000 UTC m=+66.177792387 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret") pod "global-pull-secret-syncer-2z2gr" (UID: "dbb785a9-ed6f-4480-bd9c-e190191eaf8e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.100020    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.100025    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:36.100019747 +0000 UTC m=+66.177812528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:29:04.100030 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.100034    2564 projected.go:196] Error preparing data for projected volume kube-api-access-gw42p for pod openshift-network-diagnostics/network-check-target-p2j96: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:29:04.100396 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:04.100078    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p podName:50ceb82a-401a-4b46-854e-de72397e5bc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:36.100062404 +0000 UTC m=+66.177855189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw42p" (UniqueName: "kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p") pod "network-check-target-p2j96" (UID: "50ceb82a-401a-4b46-854e-de72397e5bc9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:29:05.197366 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.197293    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeReady"
Jul 15 19:29:05.197767 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.197411    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 19:29:05.228710 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.228682    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56699b974f-hmvkq"]
Jul 15 19:29:05.230801 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.230787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.232806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.232780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 19:29:05.232917 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.232830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 19:29:05.232917 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.232852    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4j7lw\""
Jul 15 19:29:05.233083 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.233069    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 19:29:05.238640 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.238615    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 19:29:05.240091 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.240070    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rktvs"]
Jul 15 19:29:05.242046 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.242023    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.242164 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.242124    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7pcjt"]
Jul 15 19:29:05.243702 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.243680    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.243950 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.243930    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 19:29:05.244052 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.244038    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 19:29:05.244099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.244087    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9zc59\""
Jul 15 19:29:05.246015 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.245749    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 19:29:05.246135 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.246099    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 19:29:05.246135 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.246110    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jtlc5\""
Jul 15 19:29:05.246332 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.246138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 19:29:05.246556 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.246534    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56699b974f-hmvkq"]
Jul 15 19:29:05.252034 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.252013    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rktvs"]
Jul 15 19:29:05.254963 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.254945    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7pcjt"]
Jul 15 19:29:05.307683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-bound-sa-token\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.307683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46844f3e-e443-47a1-a487-82d143fed4c9-ca-trust-extracted\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.307917 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-installation-pull-secrets\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.307917 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307784    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-trusted-ca\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.307917 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.308027 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-image-registry-private-configuration\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.308027 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.307951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6slv\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-kube-api-access-d6slv\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.308027 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.308011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-registry-certificates\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408507 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-installation-pull-secrets\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408507 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.408711 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-trusted-ca\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408711 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408711 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-image-registry-private-configuration\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408711 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e628e49a-bcdb-46c6-91d2-e6942e8b1eb5-cert\") pod \"ingress-canary-7pcjt\" (UID: \"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5\") " pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6slv\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-kube-api-access-d6slv\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.408640    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-registry-certificates\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.408755    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408773    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxf7r\" (UniqueName: \"kubernetes.io/projected/e628e49a-bcdb-46c6-91d2-e6942e8b1eb5-kube-api-access-gxf7r\") pod \"ingress-canary-7pcjt\" (UID: \"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5\") " pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d15779c7-6817-42eb-98ad-71fee2675631-config-volume\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.408828    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:05.908799105 +0000 UTC m=+35.986591905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-bound-sa-token\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.408898 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46844f3e-e443-47a1-a487-82d143fed4c9-ca-trust-extracted\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.409348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15779c7-6817-42eb-98ad-71fee2675631-tmp-dir\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.409348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.408950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chnck\" (UniqueName: \"kubernetes.io/projected/d15779c7-6817-42eb-98ad-71fee2675631-kube-api-access-chnck\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.409348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.409316    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46844f3e-e443-47a1-a487-82d143fed4c9-ca-trust-extracted\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.409446 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.409346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-registry-certificates\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.409519 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.409501    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-trusted-ca\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.413870 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.413840    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-installation-pull-secrets\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.413870 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.413848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-image-registry-private-configuration\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.416584 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.416560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-bound-sa-token\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.416666 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.416561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6slv\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-kube-api-access-d6slv\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.486769 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.486736    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:05.486931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.486753    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:29:05.486931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.486874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:05.489135 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.489113    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:29:05.489253 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.489145    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:29:05.489253 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.489156    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:29:05.489455 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.489439    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5jvdh\""
Jul 15 19:29:05.489589 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.489578    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t5v82\""
Jul 15 19:29:05.489622 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.489604    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:29:05.509702 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.509680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxf7r\" (UniqueName: \"kubernetes.io/projected/e628e49a-bcdb-46c6-91d2-e6942e8b1eb5-kube-api-access-gxf7r\") pod \"ingress-canary-7pcjt\" (UID: \"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5\") " pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.509806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.509712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d15779c7-6817-42eb-98ad-71fee2675631-config-volume\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.509806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.509746    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15779c7-6817-42eb-98ad-71fee2675631-tmp-dir\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.509806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.509772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chnck\" (UniqueName: \"kubernetes.io/projected/d15779c7-6817-42eb-98ad-71fee2675631-kube-api-access-chnck\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.509959 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.509825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.509959 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.509880    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e628e49a-bcdb-46c6-91d2-e6942e8b1eb5-cert\") pod \"ingress-canary-7pcjt\" (UID: \"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5\") " pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.510044 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.509994    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:05.510098 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.510057    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:06.010036681 +0000 UTC m=+36.087829462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:05.510098 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.510077    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d15779c7-6817-42eb-98ad-71fee2675631-tmp-dir\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.510256 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.510214    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d15779c7-6817-42eb-98ad-71fee2675631-config-volume\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.512427 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.512408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e628e49a-bcdb-46c6-91d2-e6942e8b1eb5-cert\") pod \"ingress-canary-7pcjt\" (UID: \"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5\") " pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.518172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.518142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chnck\" (UniqueName: \"kubernetes.io/projected/d15779c7-6817-42eb-98ad-71fee2675631-kube-api-access-chnck\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:05.518437 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.518418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxf7r\" (UniqueName: \"kubernetes.io/projected/e628e49a-bcdb-46c6-91d2-e6942e8b1eb5-kube-api-access-gxf7r\") pod \"ingress-canary-7pcjt\" (UID: \"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5\") " pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.558559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.558512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7pcjt"
Jul 15 19:29:05.702438 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.702407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7pcjt"]
Jul 15 19:29:05.707189 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:29:05.707161    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode628e49a_bcdb_46c6_91d2_e6942e8b1eb5.slice/crio-d7900556be1dbfd069482592b6f933915c9d132e89c383f0afa51a52466db895 WatchSource:0}: Error finding container d7900556be1dbfd069482592b6f933915c9d132e89c383f0afa51a52466db895: Status 404 returned error can't find the container with id d7900556be1dbfd069482592b6f933915c9d132e89c383f0afa51a52466db895
Jul 15 19:29:05.906703 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.906585    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:05.914596 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:05.914561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:05.914739 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.914709    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:05.914739 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.914724    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:05.914818 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:05.914768    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:06.914754995 +0000 UTC m=+36.992547776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:06.012890 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:06.012848    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:06.015837 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:06.015818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:06.015935 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:06.015924    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:06.015985 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:06.015975    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:07.015962128 +0000 UTC m=+37.093754907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:06.677420 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:06.677380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7pcjt" event={"ID":"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5","Type":"ContainerStarted","Data":"d7900556be1dbfd069482592b6f933915c9d132e89c383f0afa51a52466db895"}
Jul 15 19:29:06.923657 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:06.923626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:06.923847 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:06.923803    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:06.923847 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:06.923828    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:06.923966 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:06.923904    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:08.923883123 +0000 UTC m=+39.001675915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:07.025027 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:07.024986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:07.025215 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:07.025175    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:07.025312 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:07.025301    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:09.025277832 +0000 UTC m=+39.103070618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:07.955046 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:07.955008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:08.333246 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:08.333138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:08.433122 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:08.433092    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:08.940080 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:08.940035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:08.940301 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:08.940202    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:08.940301 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:08.940248    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:08.940412 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:08.940322    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:12.940300911 +0000 UTC m=+43.018093705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:09.040754 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:09.040723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:09.041190 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:09.040897    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:09.041190 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:09.040960    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:13.040944651 +0000 UTC m=+43.118737432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:10.690167 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:10.689925    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcdbed1e-b25a-49f4-8957-e6687fcaf0f5" containerID="3c3a266cf859f45df4eeed240dc8d2149b9c6b452e54bfda80c7c7e4aeadc76b" exitCode=0
Jul 15 19:29:10.690167 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:10.689986    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerDied","Data":"3c3a266cf859f45df4eeed240dc8d2149b9c6b452e54bfda80c7c7e4aeadc76b"}
Jul 15 19:29:10.691649 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:10.691624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7pcjt" event={"ID":"e628e49a-bcdb-46c6-91d2-e6942e8b1eb5","Type":"ContainerStarted","Data":"4c272d0526a60c34beb56038832b0f88adbf82575a9ada006eb9ded8e95e7e39"}
Jul 15 19:29:10.725340 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:10.725291    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7pcjt" podStartSLOduration=1.176737551 podStartE2EDuration="5.725273047s" totalImagesPullingTime="4.548535496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:10.724778934 +0000 UTC m=+40.802571736" watchObservedRunningTime="2026-07-15 19:29:10.725273047 +0000 UTC m=+40.803065854"
Jul 15 19:29:11.495682 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:11.495651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:11.597869 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:11.597831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:11.697308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:11.697277    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcdbed1e-b25a-49f4-8957-e6687fcaf0f5" containerID="97e3734e1a7d94ff4884097c9cc59d5857425381d27f6fc249c1ad4962273e98" exitCode=0
Jul 15 19:29:11.697902 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:11.697365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerDied","Data":"97e3734e1a7d94ff4884097c9cc59d5857425381d27f6fc249c1ad4962273e98"}
Jul 15 19:29:12.702554 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:12.702520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" event={"ID":"fcdbed1e-b25a-49f4-8957-e6687fcaf0f5","Type":"ContainerStarted","Data":"5eab0920d1fa1979546fe2403585846215e9a1d114bcedff6df7985a64f512ca"}
Jul 15 19:29:12.721987 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:12.721935    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ztbhl" podStartSLOduration=9.644101922 podStartE2EDuration="42.721908533s" totalImagesPullingTime="33.077806611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:12.720695309 +0000 UTC m=+42.798488109" watchObservedRunningTime="2026-07-15 19:29:12.721908533 +0000 UTC m=+42.799701335"
Jul 15 19:29:12.841095 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:12.841056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:12.896631 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:12.896592    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:12.978246 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:12.978162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:12.978356 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:12.978293    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:12.978356 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:12.978306    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:12.978419 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:12.978368    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:20.978350275 +0000 UTC m=+51.056143075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:13.079517 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:13.079484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:13.079648 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:13.079625    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:13.079725 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:13.079714    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:21.0796951 +0000 UTC m=+51.157487879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:18.054413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:18.054377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:18.148837 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:18.148795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:19.201349 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:19.201313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:19.252786 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:19.252753    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:19.323688 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:19.323653    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:19.373895 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:19.373867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:20.059501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:20.059468    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:20.158290 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:20.158257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:21.037644 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:21.037604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:21.038027 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:21.037768    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:21.038027 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:21.037789    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:21.038027 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:21.037859    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:37.03784348 +0000 UTC m=+67.115636287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:21.137982 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:21.137945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:21.138166 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:21.138108    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:21.138265 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:21.138198    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:37.138177397 +0000 UTC m=+67.215970194 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:30.674564 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:30.674537    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlqrq"
Jul 15 19:29:30.826343 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:30.826286    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:30.921667 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:30.921627    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:36.139318 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.139279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:36.139719 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.139333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:29:36.139719 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.139420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:36.141617 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.141600    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:29:36.141716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.141638    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:29:36.141716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.141680    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:29:36.150247 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:36.150210    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 19:29:36.150336 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:36.150280    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs podName:88f9a415-fea9-462f-83bf-e457ab604332 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:40.150264623 +0000 UTC m=+130.228057402 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs") pod "network-metrics-daemon-9pclt" (UID: "88f9a415-fea9-462f-83bf-e457ab604332") : secret "metrics-daemon-secret" not found
Jul 15 19:29:36.151940 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.151925    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:29:36.154186 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.154170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbb785a9-ed6f-4480-bd9c-e190191eaf8e-original-pull-secret\") pod \"global-pull-secret-syncer-2z2gr\" (UID: \"dbb785a9-ed6f-4480-bd9c-e190191eaf8e\") " pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:36.163134 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.163110    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw42p\" (UniqueName: \"kubernetes.io/projected/50ceb82a-401a-4b46-854e-de72397e5bc9-kube-api-access-gw42p\") pod \"network-check-target-p2j96\" (UID: \"50ceb82a-401a-4b46-854e-de72397e5bc9\") " pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:36.399478 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.399398    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5jvdh\""
Jul 15 19:29:36.401889 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.401872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z2gr"
Jul 15 19:29:36.407802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.407785    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:36.558307 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.558275    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p2j96"]
Jul 15 19:29:36.561180 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:29:36.561154    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ceb82a_401a_4b46_854e_de72397e5bc9.slice/crio-db915ff4cf1be08c69eb804c51c7333cdd2530d8b6b0c1dbaa3b30477a850c43 WatchSource:0}: Error finding container db915ff4cf1be08c69eb804c51c7333cdd2530d8b6b0c1dbaa3b30477a850c43: Status 404 returned error can't find the container with id db915ff4cf1be08c69eb804c51c7333cdd2530d8b6b0c1dbaa3b30477a850c43
Jul 15 19:29:36.570068 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.570043    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z2gr"]
Jul 15 19:29:36.573522 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:29:36.573494    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb785a9_ed6f_4480_bd9c_e190191eaf8e.slice/crio-cb1d22055d72cc2e9b47b12fc63de2a51429a7b377f9785ad267a72024e1d216 WatchSource:0}: Error finding container cb1d22055d72cc2e9b47b12fc63de2a51429a7b377f9785ad267a72024e1d216: Status 404 returned error can't find the container with id cb1d22055d72cc2e9b47b12fc63de2a51429a7b377f9785ad267a72024e1d216
Jul 15 19:29:36.760008 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.759970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p2j96" event={"ID":"50ceb82a-401a-4b46-854e-de72397e5bc9","Type":"ContainerStarted","Data":"db915ff4cf1be08c69eb804c51c7333cdd2530d8b6b0c1dbaa3b30477a850c43"}
Jul 15 19:29:36.761034 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:36.761012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z2gr" event={"ID":"dbb785a9-ed6f-4480-bd9c-e190191eaf8e","Type":"ContainerStarted","Data":"cb1d22055d72cc2e9b47b12fc63de2a51429a7b377f9785ad267a72024e1d216"}
Jul 15 19:29:37.045372 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:37.045282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:29:37.045537 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:37.045431    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:29:37.045537 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:37.045446    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56699b974f-hmvkq: secret "image-registry-tls" not found
Jul 15 19:29:37.045537 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:37.045514    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls podName:46844f3e-e443-47a1-a487-82d143fed4c9 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:09.045493857 +0000 UTC m=+99.123286637 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls") pod "image-registry-56699b974f-hmvkq" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9") : secret "image-registry-tls" not found
Jul 15 19:29:37.145613 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:37.145575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:29:37.146053 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:37.145747    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:29:37.146053 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:37.145830    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls podName:d15779c7-6817-42eb-98ad-71fee2675631 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:09.145809645 +0000 UTC m=+99.223602430 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls") pod "dns-default-rktvs" (UID: "d15779c7-6817-42eb-98ad-71fee2675631") : secret "dns-default-metrics-tls" not found
Jul 15 19:29:38.535525 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.535481    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-fp5ps"]
Jul 15 19:29:38.538822 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.538800    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"]
Jul 15 19:29:38.538963 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.538942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.540824 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.540798    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f9ccbbdb-bqbnh"]
Jul 15 19:29:38.540951 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.540929    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:38.541503 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.541461    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9zmvk\""
Jul 15 19:29:38.541731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.541708    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 15 19:29:38.541852 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.541830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 15 19:29:38.542329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.542310    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 19:29:38.542697 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.542640    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 19:29:38.542697 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.542644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.543636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.543504    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 15 19:29:38.543636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.543552    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:29:38.543789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.543639    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9xgsz\""
Jul 15 19:29:38.543789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.543504    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 15 19:29:38.545092 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.544953    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 15 19:29:38.545092 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.544985    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 15 19:29:38.545092 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.545033    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 19:29:38.545092 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.545095    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 15 19:29:38.545361 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.545153    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cddhz\""
Jul 15 19:29:38.545395 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.545362    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 15 19:29:38.545606 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.545590    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 19:29:38.548599 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.548581    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 15 19:29:38.554058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.554038    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-fp5ps"]
Jul 15 19:29:38.554181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.554160    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"]
Jul 15 19:29:38.555266 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.555247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f9ccbbdb-bqbnh"]
Jul 15 19:29:38.556893 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.556877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-stats-auth\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.556977 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.556900    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tsfn\" (UniqueName: \"kubernetes.io/projected/22a24f49-4050-4490-aaaa-0a317701ded1-kube-api-access-6tsfn\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.556977 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.556930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7f26\" (UniqueName: \"kubernetes.io/projected/88b5f2b2-0bfd-4c38-a46e-27d15779967e-kube-api-access-v7f26\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:38.556977 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.556949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.557094 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22a24f49-4050-4490-aaaa-0a317701ded1-serving-cert\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.557094 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557064    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a24f49-4050-4490-aaaa-0a317701ded1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.557191 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.557191 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557138    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:38.557191 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22a24f49-4050-4490-aaaa-0a317701ded1-tmp\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.557191 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/22a24f49-4050-4490-aaaa-0a317701ded1-snapshots\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.557390 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4ppp\" (UniqueName: \"kubernetes.io/projected/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-kube-api-access-c4ppp\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.557390 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-default-certificate\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.557390 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.557364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a24f49-4050-4490-aaaa-0a317701ded1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.657843 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.657803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-stats-auth\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.657843 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.657840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tsfn\" (UniqueName: \"kubernetes.io/projected/22a24f49-4050-4490-aaaa-0a317701ded1-kube-api-access-6tsfn\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658112 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7f26\" (UniqueName: \"kubernetes.io/projected/88b5f2b2-0bfd-4c38-a46e-27d15779967e-kube-api-access-v7f26\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:38.658112 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.658112 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22a24f49-4050-4490-aaaa-0a317701ded1-serving-cert\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a24f49-4050-4490-aaaa-0a317701ded1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.658329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:38.658329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22a24f49-4050-4490-aaaa-0a317701ded1-tmp\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658329 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:38.658264    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:39.158243429 +0000 UTC m=+69.236036209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : configmap references non-existent config key: service-ca.crt
Jul 15 19:29:38.658329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/22a24f49-4050-4490-aaaa-0a317701ded1-snapshots\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4ppp\" (UniqueName: \"kubernetes.io/projected/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-kube-api-access-c4ppp\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.658544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-default-certificate\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.658544 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:38.658423    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:29:38.658544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658464    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a24f49-4050-4490-aaaa-0a317701ded1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658544 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:38.658482    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:39.158465339 +0000 UTC m=+69.236258121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : secret "router-metrics-certs-default" not found
Jul 15 19:29:38.658738 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:38.658585    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 19:29:38.658738 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:38.658648    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls podName:88b5f2b2-0bfd-4c38-a46e-27d15779967e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:39.158627227 +0000 UTC m=+69.236420022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls") pod "cluster-samples-operator-65db858894-59jnv" (UID: "88b5f2b2-0bfd-4c38-a46e-27d15779967e") : secret "samples-operator-tls" not found
Jul 15 19:29:38.658738 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22a24f49-4050-4490-aaaa-0a317701ded1-tmp\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.658942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.658910    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a24f49-4050-4490-aaaa-0a317701ded1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.659200 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.659175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/22a24f49-4050-4490-aaaa-0a317701ded1-snapshots\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.660105 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.660051    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22a24f49-4050-4490-aaaa-0a317701ded1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.661710 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.661688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22a24f49-4050-4490-aaaa-0a317701ded1-serving-cert\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.662400 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.662377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-stats-auth\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.663109 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.663084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-default-certificate\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.672538 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.672485    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tsfn\" (UniqueName: \"kubernetes.io/projected/22a24f49-4050-4490-aaaa-0a317701ded1-kube-api-access-6tsfn\") pod \"insights-operator-68d9dd94cf-fp5ps\" (UID: \"22a24f49-4050-4490-aaaa-0a317701ded1\") " pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:38.674790 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.674748    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7f26\" (UniqueName: \"kubernetes.io/projected/88b5f2b2-0bfd-4c38-a46e-27d15779967e-kube-api-access-v7f26\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:38.674915 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.674810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4ppp\" (UniqueName: \"kubernetes.io/projected/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-kube-api-access-c4ppp\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:38.705105 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.705063    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:38.768170 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.768134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:38.850874 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:38.850782    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps"
Jul 15 19:29:39.162628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.162525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:39.162628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.162573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:39.162628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.162599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:39.162912 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:39.162728    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 19:29:39.162912 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:39.162729    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:29:39.162912 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:39.162743    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:40.162718831 +0000 UTC m=+70.240511622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : configmap references non-existent config key: service-ca.crt
Jul 15 19:29:39.162912 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:39.162801    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls podName:88b5f2b2-0bfd-4c38-a46e-27d15779967e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:40.162785278 +0000 UTC m=+70.240578070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls") pod "cluster-samples-operator-65db858894-59jnv" (UID: "88b5f2b2-0bfd-4c38-a46e-27d15779967e") : secret "samples-operator-tls" not found
Jul 15 19:29:39.162912 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:39.162818    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:40.162807815 +0000 UTC m=+70.240600614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : secret "router-metrics-certs-default" not found
Jul 15 19:29:39.344155 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.344120    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-fp5ps"]
Jul 15 19:29:39.347929 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:29:39.347902    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a24f49_4050_4490_aaaa_0a317701ded1.slice/crio-1f7e4393b7a94eb18cea877e27fa679f35ff949eae7d327a8873bb2270a84feb WatchSource:0}: Error finding container 1f7e4393b7a94eb18cea877e27fa679f35ff949eae7d327a8873bb2270a84feb: Status 404 returned error can't find the container with id 1f7e4393b7a94eb18cea877e27fa679f35ff949eae7d327a8873bb2270a84feb
Jul 15 19:29:39.769727 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.769689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps" event={"ID":"22a24f49-4050-4490-aaaa-0a317701ded1","Type":"ContainerStarted","Data":"1f7e4393b7a94eb18cea877e27fa679f35ff949eae7d327a8873bb2270a84feb"}
Jul 15 19:29:39.771337 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.771309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p2j96" event={"ID":"50ceb82a-401a-4b46-854e-de72397e5bc9","Type":"ContainerStarted","Data":"b3214a741eb1b51a23c66252981e00a45f63299baa75f45e1dde5c52e4ffde6b"}
Jul 15 19:29:39.771472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.771455    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:29:39.786228 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:39.786178    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p2j96" podStartSLOduration=67.095027061 podStartE2EDuration="1m9.786160762s" totalImagesPullingTime="2.691133701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:39.784850139 +0000 UTC m=+69.862642941" watchObservedRunningTime="2026-07-15 19:29:39.786160762 +0000 UTC m=+69.863953564"
Jul 15 19:29:40.020971 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.020885    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:40.121999 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.121955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:40.172043 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.172002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:40.172043 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.172042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:40.172354 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.172066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:40.172354 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:40.172258    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 19:29:40.172354 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:40.172315    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls podName:88b5f2b2-0bfd-4c38-a46e-27d15779967e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:42.172295884 +0000 UTC m=+72.250088685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls") pod "cluster-samples-operator-65db858894-59jnv" (UID: "88b5f2b2-0bfd-4c38-a46e-27d15779967e") : secret "samples-operator-tls" not found
Jul 15 19:29:40.172674 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:40.172659    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:42.172647669 +0000 UTC m=+72.250440451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : configmap references non-existent config key: service-ca.crt
Jul 15 19:29:40.172744 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:40.172725    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:29:40.172793 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:40.172759    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:42.172749019 +0000 UTC m=+72.250541798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : secret "router-metrics-certs-default" not found
Jul 15 19:29:40.396385 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.396306    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:40.448210 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.448174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:40.792120 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.792081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:40.841888 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:40.841844    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:41.049580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:41.049490    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:41.150653 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:41.150613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:41.778645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:41.778547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps" event={"ID":"22a24f49-4050-4490-aaaa-0a317701ded1","Type":"ContainerStarted","Data":"e1d2984375cb197be2c852a6e6aad319dadf629f4b3cf1d69321ed50ffe60070"}
Jul 15 19:29:42.191896 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:42.191804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:42.191896 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:42.191864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:42.192358 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:42.191897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:42.192358 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:42.191954    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:46.191930194 +0000 UTC m=+76.269722975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : configmap references non-existent config key: service-ca.crt
Jul 15 19:29:42.192358 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:42.192014    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:29:42.192358 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:42.192015    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 19:29:42.192358 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:42.192055    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:46.19204391 +0000 UTC m=+76.269836690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : secret "router-metrics-certs-default" not found
Jul 15 19:29:42.192358 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:42.192100    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls podName:88b5f2b2-0bfd-4c38-a46e-27d15779967e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:46.192090714 +0000 UTC m=+76.269883501 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls") pod "cluster-samples-operator-65db858894-59jnv" (UID: "88b5f2b2-0bfd-4c38-a46e-27d15779967e") : secret "samples-operator-tls" not found
Jul 15 19:29:42.819118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:42.819061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:42.918083 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:42.918046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:43.019450 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.019411    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:43.347542 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.347463    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps" podStartSLOduration=3.208774795 podStartE2EDuration="5.34744019s" totalImagesPullingTime="2.138665395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:41.79411327 +0000 UTC m=+71.871906071" watchObservedRunningTime="2026-07-15 19:29:43.34744019 +0000 UTC m=+73.425232994"
Jul 15 19:29:43.348306 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.348277    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6brrv"]
Jul 15 19:29:43.362761 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.362723    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.363472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.363449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6brrv"]
Jul 15 19:29:43.364728 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.364706    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wwfmj\""
Jul 15 19:29:43.364862 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.364845    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 19:29:43.364931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.364847    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 19:29:43.396731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.396696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:43.406793 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.406755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.406793 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.406793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59dpc\" (UniqueName: \"kubernetes.io/projected/398023a0-b103-4631-9812-b1785776bfe6-kube-api-access-59dpc\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.407002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.406918    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/398023a0-b103-4631-9812-b1785776bfe6-crio-socket\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.407002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.406952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/398023a0-b103-4631-9812-b1785776bfe6-data-volume\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.407002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.406987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.444580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.444548    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:43.507808 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.507770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.507808 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.507807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59dpc\" (UniqueName: \"kubernetes.io/projected/398023a0-b103-4631-9812-b1785776bfe6-kube-api-access-59dpc\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.508020 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.507885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/398023a0-b103-4631-9812-b1785776bfe6-crio-socket\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.508020 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:43.507901    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:29:43.508020 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.507919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/398023a0-b103-4631-9812-b1785776bfe6-data-volume\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.508020 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:43.507966    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:44.007947502 +0000 UTC m=+74.085740298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : configmap "kube-rbac-proxy" not found
Jul 15 19:29:43.508020 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.507984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.508020 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.507999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/398023a0-b103-4631-9812-b1785776bfe6-crio-socket\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.508289 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:43.508081    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:43.508289 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:43.508120    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:44.008106667 +0000 UTC m=+74.085899447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:43.508357 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.508299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/398023a0-b103-4631-9812-b1785776bfe6-data-volume\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:43.517986 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:43.517962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59dpc\" (UniqueName: \"kubernetes.io/projected/398023a0-b103-4631-9812-b1785776bfe6-kube-api-access-59dpc\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:44.012557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.012522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:44.012785 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.012610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:44.012785 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:44.012651    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:29:44.012785 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:44.012729    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:44.012785 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:44.012732    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:45.012715826 +0000 UTC m=+75.090508628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : configmap "kube-rbac-proxy" not found
Jul 15 19:29:44.012785 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:44.012775    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:45.012764759 +0000 UTC m=+75.090557537 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:44.456631 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.456598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:44.558075 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.558038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:44.787876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.787795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z2gr" event={"ID":"dbb785a9-ed6f-4480-bd9c-e190191eaf8e","Type":"ContainerStarted","Data":"a56eeb2ea6cb780f94d78cafbb92f43d47d80fc1979d303643467099017ec402"}
Jul 15 19:29:44.801387 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.801330    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2z2gr" podStartSLOduration=67.69831736 podStartE2EDuration="1m14.801314866s" totalImagesPullingTime="7.102997506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:44.800087391 +0000 UTC m=+74.877880191" watchObservedRunningTime="2026-07-15 19:29:44.801314866 +0000 UTC m=+74.879107666"
Jul 15 19:29:44.897736 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:44.897697    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:45.020760 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:45.020724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:45.020928 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:45.020791    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:45.020928 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:45.020876    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:29:45.020928 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:45.020887    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:45.020928 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:45.020929    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:47.020916276 +0000 UTC m=+77.098709055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : configmap "kube-rbac-proxy" not found
Jul 15 19:29:45.021066 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:45.020942    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:47.020936415 +0000 UTC m=+77.098729194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:45.087759 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:45.087672    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:45.188234 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:45.188182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:45.240175 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:45.240142    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:45.409374 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:45.409288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:46.153261 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:46.153211    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jx4q7_4e405546-e58e-4c94-a49f-c6c18b7a28d6/dns-node-resolver/0.log"
Jul 15 19:29:46.233930 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:46.233896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:46.233930 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:46.233935    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:46.234129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:46.233958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:46.234129 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:46.234054    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 19:29:46.234129 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:46.234056    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:29:46.234129 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:46.234080    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:54.234063784 +0000 UTC m=+84.311856563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : configmap references non-existent config key: service-ca.crt
Jul 15 19:29:46.234129 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:46.234102    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls podName:88b5f2b2-0bfd-4c38-a46e-27d15779967e nodeName:}" failed. No retries permitted until 2026-07-15 19:29:54.234095054 +0000 UTC m=+84.311887832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls") pod "cluster-samples-operator-65db858894-59jnv" (UID: "88b5f2b2-0bfd-4c38-a46e-27d15779967e") : secret "samples-operator-tls" not found
Jul 15 19:29:46.234129 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:46.234112    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:54.234106935 +0000 UTC m=+84.311899714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : secret "router-metrics-certs-default" not found
Jul 15 19:29:46.779411 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:46.779365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:46.835627 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:46.835594    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:47.039173 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:47.039084    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:47.039360 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:47.039209    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:29:47.039360 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:47.039294    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:51.039277517 +0000 UTC m=+81.117070299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : configmap "kube-rbac-proxy" not found
Jul 15 19:29:47.039360 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:47.039324    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:47.039360 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:47.039213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:47.039533 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:47.039378    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:51.039360799 +0000 UTC m=+81.117153582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:47.152970 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:47.152950    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k29ss_dcb8fe74-76fa-416d-9203-992448ca4ddd/node-ca/0.log"
Jul 15 19:29:48.157091 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:48.157048    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7pcjt_e628e49a-bcdb-46c6-91d2-e6942e8b1eb5/serve-healthcheck-canary/0.log"
Jul 15 19:29:48.765978 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:48.765941    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:51.070519 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:51.070480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:51.070885 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:51.070552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:51.070885 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:51.070633    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:51.070885 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:51.070653    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:29:51.070885 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:51.070709    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:59.070692597 +0000 UTC m=+89.148485381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:51.070885 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:51.070726    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:29:59.070718165 +0000 UTC m=+89.148510944 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : configmap "kube-rbac-proxy" not found
Jul 15 19:29:51.717096 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:51.716963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:52.367332 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:52.367296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:52.471890 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:52.471855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:52.573966 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:52.573935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:52.668239 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:52.668136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:52.721310 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:52.721272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:53.461707 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:53.461673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:53.864388 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:53.864294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:54.259760 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.259726    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:54.296858 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.296794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:54.297010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.296975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:54.297049 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.297014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:29:54.297120 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:54.297104    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 15 19:29:54.297174 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:54.297156    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:10.297134242 +0000 UTC m=+100.374927040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : configmap references non-existent config key: service-ca.crt
Jul 15 19:29:54.297216 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:54.297184    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs podName:299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:10.297176821 +0000 UTC m=+100.374969600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs") pod "router-default-7f9ccbbdb-bqbnh" (UID: "299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2") : secret "router-metrics-certs-default" not found
Jul 15 19:29:54.299423 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.299396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88b5f2b2-0bfd-4c38-a46e-27d15779967e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-59jnv\" (UID: \"88b5f2b2-0bfd-4c38-a46e-27d15779967e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:54.458404 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.458358    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:54.460194 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.460176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"
Jul 15 19:29:54.585575 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.585543    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv"]
Jul 15 19:29:54.814754 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:54.814669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv" event={"ID":"88b5f2b2-0bfd-4c38-a46e-27d15779967e","Type":"ContainerStarted","Data":"bbcbf00d9bc8ee6ec424eec162320f96f3d5174a71db1703a49524f309438c40"}
Jul 15 19:29:55.062144 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:55.062105    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:55.171407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:55.171330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:55.265765 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:55.265728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:55.866010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:55.865977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:56.002903 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.002821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:56.104664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.104629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:56.161509 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.161420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:56.513863 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.513827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:56.763975 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.763941    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:56.825306 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.825262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv" event={"ID":"88b5f2b2-0bfd-4c38-a46e-27d15779967e","Type":"ContainerStarted","Data":"8ad04da948c23747c61fcc1611600303d0d7496004de4cf01e98bd4ea67e2040"}
Jul 15 19:29:56.825475 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.825331    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv" event={"ID":"88b5f2b2-0bfd-4c38-a46e-27d15779967e","Type":"ContainerStarted","Data":"a12f1fe95c7b74505b3bf3d84d79e5036b63992e92d20239197350a0e61f112c"}
Jul 15 19:29:56.842548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:56.842498    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-59jnv" podStartSLOduration=17.082693716 podStartE2EDuration="18.84248167s" totalImagesPullingTime="1.759787954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:56.841357941 +0000 UTC m=+86.919150743" watchObservedRunningTime="2026-07-15 19:29:56.84248167 +0000 UTC m=+86.920274481"
Jul 15 19:29:57.060250 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:57.060138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:57.459095 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:57.459056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:57.517050 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:57.517013    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:57.859543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:57.859429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:58.261796 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:58.261765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:58.458530 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:58.458492    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:58.867428 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:58.867389    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:58.972234 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:58.972186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:59.140386 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:59.140294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:59.140386 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:59.140366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:59.140563 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:59.140461    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:59.140563 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:29:59.140527    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls podName:398023a0-b103-4631-9812-b1785776bfe6 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:15.140512007 +0000 UTC m=+105.218304789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6brrv" (UID: "398023a0-b103-4631-9812-b1785776bfe6") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:29:59.140879 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:59.140848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/398023a0-b103-4631-9812-b1785776bfe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:29:59.659360 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:29:59.659323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:00.063149 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:00.063117    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:00.260953 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:00.260915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:00.458595 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:00.458567    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:00.661573 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:00.661536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:01.213681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:01.213648    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:01.314631 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:01.314598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:01.379023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:01.378979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:01.477709 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:01.477614    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:01.714456 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:01.714428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:01.860096 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:01.860008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:03.078269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:03.078216    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:04.269889 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:04.269852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:04.321012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:04.320975    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:06.913483 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:06.913446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:06.960570 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:06.960531    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:08.142619 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.142586    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-d7hbg"]
Jul 15 19:30:08.145580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.145555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:08.147561 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.147531    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 19:30:08.147561 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.147560    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 19:30:08.147747 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.147593    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wvqmr\""
Jul 15 19:30:08.158196 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.158173    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-d7hbg"]
Jul 15 19:30:08.186775 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.186743    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:08.284819 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.284781    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:08.319947 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.319897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrclc\" (UniqueName: \"kubernetes.io/projected/b05d4d84-59fb-4855-83d2-5f285ffcea40-kube-api-access-lrclc\") pod \"downloads-68c9cbb497-d7hbg\" (UID: \"b05d4d84-59fb-4855-83d2-5f285ffcea40\") " pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:08.387377 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.387340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:08.421288 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.421207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrclc\" (UniqueName: \"kubernetes.io/projected/b05d4d84-59fb-4855-83d2-5f285ffcea40-kube-api-access-lrclc\") pod \"downloads-68c9cbb497-d7hbg\" (UID: \"b05d4d84-59fb-4855-83d2-5f285ffcea40\") " pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:08.429527 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.429494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrclc\" (UniqueName: \"kubernetes.io/projected/b05d4d84-59fb-4855-83d2-5f285ffcea40-kube-api-access-lrclc\") pod \"downloads-68c9cbb497-d7hbg\" (UID: \"b05d4d84-59fb-4855-83d2-5f285ffcea40\") " pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:08.454514 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.454485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:08.483893 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.483856    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:08.578269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.578114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-d7hbg"]
Jul 15 19:30:08.580755 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:08.580723    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05d4d84_59fb_4855_83d2_5f285ffcea40.slice/crio-4943bceed039540c46ab91a5d82a02e27be40076eb8def0ecd90b2d945defe6f WatchSource:0}: Error finding container 4943bceed039540c46ab91a5d82a02e27be40076eb8def0ecd90b2d945defe6f: Status 404 returned error can't find the container with id 4943bceed039540c46ab91a5d82a02e27be40076eb8def0ecd90b2d945defe6f
Jul 15 19:30:08.865005 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:08.864971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-d7hbg" event={"ID":"b05d4d84-59fb-4855-83d2-5f285ffcea40","Type":"ContainerStarted","Data":"4943bceed039540c46ab91a5d82a02e27be40076eb8def0ecd90b2d945defe6f"}
Jul 15 19:30:09.126946 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.126848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:09.129462 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.129443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"image-registry-56699b974f-hmvkq\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") " pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:09.144348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.144328    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4j7lw\""
Jul 15 19:30:09.153071 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.153049    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:09.227552 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.227519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:30:09.230786 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.230756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d15779c7-6817-42eb-98ad-71fee2675631-metrics-tls\") pod \"dns-default-rktvs\" (UID: \"d15779c7-6817-42eb-98ad-71fee2675631\") " pod="openshift-dns/dns-default-rktvs"
Jul 15 19:30:09.296370 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.296334    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56699b974f-hmvkq"]
Jul 15 19:30:09.299794 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:09.299751    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46844f3e_e443_47a1_a487_82d143fed4c9.slice/crio-3e4be9fd2d49a5c041db657a82f3a10bc2b26fc7c2b65177c11ea42dd2b23c82 WatchSource:0}: Error finding container 3e4be9fd2d49a5c041db657a82f3a10bc2b26fc7c2b65177c11ea42dd2b23c82: Status 404 returned error can't find the container with id 3e4be9fd2d49a5c041db657a82f3a10bc2b26fc7c2b65177c11ea42dd2b23c82
Jul 15 19:30:09.456089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.456058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9zc59\""
Jul 15 19:30:09.464713 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.464686    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rktvs"
Jul 15 19:30:09.607439 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.607410    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rktvs"]
Jul 15 19:30:09.610422 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:09.610393    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15779c7_6817_42eb_98ad_71fee2675631.slice/crio-7a9c9880e57d8db796f39025362034f63a5f31e2f1db896a1d267c84d0b888a6 WatchSource:0}: Error finding container 7a9c9880e57d8db796f39025362034f63a5f31e2f1db896a1d267c84d0b888a6: Status 404 returned error can't find the container with id 7a9c9880e57d8db796f39025362034f63a5f31e2f1db896a1d267c84d0b888a6
Jul 15 19:30:09.871320 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.871209    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rktvs" event={"ID":"d15779c7-6817-42eb-98ad-71fee2675631","Type":"ContainerStarted","Data":"7a9c9880e57d8db796f39025362034f63a5f31e2f1db896a1d267c84d0b888a6"}
Jul 15 19:30:09.873067 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.873030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" event={"ID":"46844f3e-e443-47a1-a487-82d143fed4c9","Type":"ContainerStarted","Data":"9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a"}
Jul 15 19:30:09.873185 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.873090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" event={"ID":"46844f3e-e443-47a1-a487-82d143fed4c9","Type":"ContainerStarted","Data":"3e4be9fd2d49a5c041db657a82f3a10bc2b26fc7c2b65177c11ea42dd2b23c82"}
Jul 15 19:30:09.873536 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.873503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:09.891767 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:09.891707    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" podStartSLOduration=98.891687097 podStartE2EDuration="1m38.891687097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:09.891076644 +0000 UTC m=+99.968869448" watchObservedRunningTime="2026-07-15 19:30:09.891687097 +0000 UTC m=+99.969479901"
Jul 15 19:30:10.337346 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.336796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:10.337346 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.336856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:10.337845 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.337559    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-service-ca-bundle\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:10.340152 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.340128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2-metrics-certs\") pod \"router-default-7f9ccbbdb-bqbnh\" (UID: \"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2\") " pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:10.367238 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.367188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:10.515506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.515471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f9ccbbdb-bqbnh"]
Jul 15 19:30:10.518689 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:10.518648    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299af2aa_3bc0_4ecf_bee7_639ffdbe6fe2.slice/crio-2bf5d4c705e06af2de1dcb958665dc54c73433b150435b64e195c954c230a254 WatchSource:0}: Error finding container 2bf5d4c705e06af2de1dcb958665dc54c73433b150435b64e195c954c230a254: Status 404 returned error can't find the container with id 2bf5d4c705e06af2de1dcb958665dc54c73433b150435b64e195c954c230a254
Jul 15 19:30:10.777291 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.777262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p2j96"
Jul 15 19:30:10.881972 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.881891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh" event={"ID":"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2","Type":"ContainerStarted","Data":"6ef88552ed0c28cf38da92134be5a5b08fe107711332efd2b555d936aac797d7"}
Jul 15 19:30:10.882137 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.882038    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh" event={"ID":"299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2","Type":"ContainerStarted","Data":"2bf5d4c705e06af2de1dcb958665dc54c73433b150435b64e195c954c230a254"}
Jul 15 19:30:10.899609 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:10.899551    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh" podStartSLOduration=32.899533426 podStartE2EDuration="32.899533426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:10.898885332 +0000 UTC m=+100.976678145" watchObservedRunningTime="2026-07-15 19:30:10.899533426 +0000 UTC m=+100.977326229"
Jul 15 19:30:11.368248 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:11.368198    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:11.370915 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:11.370884    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:11.885285 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:11.885252    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:11.886860 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:11.886838    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f9ccbbdb-bqbnh"
Jul 15 19:30:12.889450 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:12.889412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rktvs" event={"ID":"d15779c7-6817-42eb-98ad-71fee2675631","Type":"ContainerStarted","Data":"ec5be5ebd80d18f38e85d6fbd028a9e9b23c67a9c0f6948e635944db4a8ae6c2"}
Jul 15 19:30:12.889830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:12.889470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rktvs" event={"ID":"d15779c7-6817-42eb-98ad-71fee2675631","Type":"ContainerStarted","Data":"a07c653774e0be268bd92dc1bd71ad277f6c1f95ec6fdf9a7cfe54371b1736dd"}
Jul 15 19:30:12.889830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:12.889493    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rktvs"
Jul 15 19:30:12.903901 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:12.903835    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rktvs" podStartSLOduration=65.52417157 podStartE2EDuration="1m7.903817448s" totalImagesPullingTime="2.379645878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:12.903415666 +0000 UTC m=+102.981208525" watchObservedRunningTime="2026-07-15 19:30:12.903817448 +0000 UTC m=+102.981610250"
Jul 15 19:30:13.339102 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:13.339064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:13.391790 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:13.391750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:13.718052 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:13.718015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:13.772545 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:13.772509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:14.581025 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.580983    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:14.682173 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.682092    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:14.735503 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.735466    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"]
Jul 15 19:30:14.750970 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.750935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:14.771119 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.771088    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"]
Jul 15 19:30:14.771318 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.771195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:14.773533 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.773509    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 19:30:14.773533 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.773533    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jrkf7\""
Jul 15 19:30:14.877378 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.877287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3bc3332d-3937-4e17-af38-cb06e98059f5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j8ghn\" (UID: \"3bc3332d-3937-4e17-af38-cb06e98059f5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:14.976273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.976243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:14.978266 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.978233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3bc3332d-3937-4e17-af38-cb06e98059f5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j8ghn\" (UID: \"3bc3332d-3937-4e17-af38-cb06e98059f5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:14.981518 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:14.981492    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3bc3332d-3937-4e17-af38-cb06e98059f5-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j8ghn\" (UID: \"3bc3332d-3937-4e17-af38-cb06e98059f5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:15.082238 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.082187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:15.180235 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.179822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:30:15.182453 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.182394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/398023a0-b103-4631-9812-b1785776bfe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6brrv\" (UID: \"398023a0-b103-4631-9812-b1785776bfe6\") " pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:30:15.244054 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.244024    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"]
Jul 15 19:30:15.247725 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:15.247691    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc3332d_3937_4e17_af38_cb06e98059f5.slice/crio-856913f5a8dde93857d8bca36b8c2a2f6443090d1304e318188f9da07595878a WatchSource:0}: Error finding container 856913f5a8dde93857d8bca36b8c2a2f6443090d1304e318188f9da07595878a: Status 404 returned error can't find the container with id 856913f5a8dde93857d8bca36b8c2a2f6443090d1304e318188f9da07595878a
Jul 15 19:30:15.472723 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.472683    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6brrv"
Jul 15 19:30:15.529117 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.529082    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:15.541195 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.541160    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b98778d74-pfhxh"]
Jul 15 19:30:15.560125 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.560050    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b98778d74-pfhxh"]
Jul 15 19:30:15.560450 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.560431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.563865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.562894    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8lsmr\""
Jul 15 19:30:15.563865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.563352    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 19:30:15.566402 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.565306    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 19:30:15.566402 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.565584    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 19:30:15.566402 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.565891    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 19:30:15.569166 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.567979    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 19:30:15.652665 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.652581    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6brrv"]
Jul 15 19:30:15.655043 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:15.655014    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod398023a0_b103_4631_9812_b1785776bfe6.slice/crio-905e634b3becba728aebd72691f6857f99c97f9329cdc7603facb487a4023843 WatchSource:0}: Error finding container 905e634b3becba728aebd72691f6857f99c97f9329cdc7603facb487a4023843: Status 404 returned error can't find the container with id 905e634b3becba728aebd72691f6857f99c97f9329cdc7603facb487a4023843
Jul 15 19:30:15.683120 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.683087    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-oauth-serving-cert\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.683308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.683127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-service-ca\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.683308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.683169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-config\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.683308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.683199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd7hr\" (UniqueName: \"kubernetes.io/projected/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-kube-api-access-cd7hr\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.683308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.683242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-oauth-config\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.683308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.683284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-serving-cert\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.783830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.783745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-config\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.783830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.783785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd7hr\" (UniqueName: \"kubernetes.io/projected/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-kube-api-access-cd7hr\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.783830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.783819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-oauth-config\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.784090 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.783976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-serving-cert\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.784090 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.784030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-oauth-serving-cert\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.784090 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.784067    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-service-ca\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.784674 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.784638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-config\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.784903 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.784877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-oauth-serving-cert\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.785061 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.785038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-service-ca\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.786585 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.786564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-oauth-config\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.786878 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.786858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-serving-cert\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.791522 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.791496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd7hr\" (UniqueName: \"kubernetes.io/projected/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-kube-api-access-cd7hr\") pod \"console-b98778d74-pfhxh\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") " pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.880467 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.880431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:15.910086 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.910045    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6brrv" event={"ID":"398023a0-b103-4631-9812-b1785776bfe6","Type":"ContainerStarted","Data":"905e634b3becba728aebd72691f6857f99c97f9329cdc7603facb487a4023843"}
Jul 15 19:30:15.912471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:15.912444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn" event={"ID":"3bc3332d-3937-4e17-af38-cb06e98059f5","Type":"ContainerStarted","Data":"856913f5a8dde93857d8bca36b8c2a2f6443090d1304e318188f9da07595878a"}
Jul 15 19:30:16.045983 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:16.045900    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b98778d74-pfhxh"]
Jul 15 19:30:16.049592 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:16.049558    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7c9a99_ca74_4c17_ab04_4d6d61d18384.slice/crio-2ac56731cea37d73be6e6830f20bff2c8766b1c5e62bfc651349e3a29e4c6ebb WatchSource:0}: Error finding container 2ac56731cea37d73be6e6830f20bff2c8766b1c5e62bfc651349e3a29e4c6ebb: Status 404 returned error can't find the container with id 2ac56731cea37d73be6e6830f20bff2c8766b1c5e62bfc651349e3a29e4c6ebb
Jul 15 19:30:16.921616 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:16.921472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b98778d74-pfhxh" event={"ID":"ce7c9a99-ca74-4c17-ab04-4d6d61d18384","Type":"ContainerStarted","Data":"2ac56731cea37d73be6e6830f20bff2c8766b1c5e62bfc651349e3a29e4c6ebb"}
Jul 15 19:30:16.925557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:16.924354    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn" event={"ID":"3bc3332d-3937-4e17-af38-cb06e98059f5","Type":"ContainerStarted","Data":"1c1e9dac311fc4e81c4c26ae42a8413d9174631a37cdd49be4eacaf7761f0bef"}
Jul 15 19:30:16.925557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:16.925495    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:16.933290 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:16.933263    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn"
Jul 15 19:30:16.939651 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:16.939598    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j8ghn" podStartSLOduration=1.7361288 podStartE2EDuration="2.939579319s" totalImagesPullingTime="1.203450519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:16.937939152 +0000 UTC m=+107.015731956" watchObservedRunningTime="2026-07-15 19:30:16.939579319 +0000 UTC m=+107.017372121"
Jul 15 19:30:17.125641 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.124436    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54d9b5f5cd-88jk9"]
Jul 15 19:30:17.127421 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.126718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.137243 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.137155    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54d9b5f5cd-88jk9"]
Jul 15 19:30:17.143198 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.143160    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 19:30:17.204810 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.204773    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-oauth-config\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.205009 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.204864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bph28\" (UniqueName: \"kubernetes.io/projected/e6b11b6f-c402-4307-919c-add8df8f4538-kube-api-access-bph28\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.205009 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.204896    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-serving-cert\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.205009 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.204959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-console-config\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.205009 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.204988    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-trusted-ca-bundle\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.205253 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.205014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-oauth-serving-cert\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.205253 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.205070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-service-ca\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.305940 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.305900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bph28\" (UniqueName: \"kubernetes.io/projected/e6b11b6f-c402-4307-919c-add8df8f4538-kube-api-access-bph28\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.306118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.305953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-serving-cert\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.306118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.306007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-console-config\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.306118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.306031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-trusted-ca-bundle\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.306118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.306052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-oauth-serving-cert\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.306118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.306087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-service-ca\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.306419 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.306123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-oauth-config\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.307781 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.307726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-trusted-ca-bundle\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.308865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.308297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-oauth-serving-cert\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.308865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.308825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-service-ca\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.309633 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.309580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-console-config\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.313751 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.313706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-oauth-config\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.317817 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.317732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bph28\" (UniqueName: \"kubernetes.io/projected/e6b11b6f-c402-4307-919c-add8df8f4538-kube-api-access-bph28\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.318511 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.318489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-serving-cert\") pod \"console-54d9b5f5cd-88jk9\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") " pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.338080 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.338042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:17.389269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.389210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:17.445647 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.445612    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:17.607324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.607285    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54d9b5f5cd-88jk9"]
Jul 15 19:30:17.611370 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:17.611339    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b11b6f_c402_4307_919c_add8df8f4538.slice/crio-4649ad967ec38f9f2d6790a85bb95c1a183c81ef7af13e251e134a77ff00da83 WatchSource:0}: Error finding container 4649ad967ec38f9f2d6790a85bb95c1a183c81ef7af13e251e134a77ff00da83: Status 404 returned error can't find the container with id 4649ad967ec38f9f2d6790a85bb95c1a183c81ef7af13e251e134a77ff00da83
Jul 15 19:30:17.784311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.782809    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cv4rf"]
Jul 15 19:30:17.795806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.795085    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cv4rf"]
Jul 15 19:30:17.795806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.795212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.798344 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.798107    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 15 19:30:17.798344 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.798119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mbq2m\""
Jul 15 19:30:17.798344 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.798126    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 19:30:17.798344 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.798126    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 19:30:17.798344 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.798309    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 15 19:30:17.798669 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.798492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 19:30:17.812624 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.811850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ec833a-e136-4858-b688-9eed5924af6c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.812624 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.812019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ec833a-e136-4858-b688-9eed5924af6c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.812624 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.812083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl8m8\" (UniqueName: \"kubernetes.io/projected/d2ec833a-e136-4858-b688-9eed5924af6c-kube-api-access-cl8m8\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.812624 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.812262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2ec833a-e136-4858-b688-9eed5924af6c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.913882 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.913838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ec833a-e136-4858-b688-9eed5924af6c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.914071 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.913899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ec833a-e136-4858-b688-9eed5924af6c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.914071 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.913931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl8m8\" (UniqueName: \"kubernetes.io/projected/d2ec833a-e136-4858-b688-9eed5924af6c-kube-api-access-cl8m8\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.914071 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.913966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2ec833a-e136-4858-b688-9eed5924af6c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.915058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.914717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2ec833a-e136-4858-b688-9eed5924af6c-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.917701 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.917654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ec833a-e136-4858-b688-9eed5924af6c-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.918236 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.918145    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2ec833a-e136-4858-b688-9eed5924af6c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.924495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.924468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl8m8\" (UniqueName: \"kubernetes.io/projected/d2ec833a-e136-4858-b688-9eed5924af6c-kube-api-access-cl8m8\") pod \"prometheus-operator-64785b6675-cv4rf\" (UID: \"d2ec833a-e136-4858-b688-9eed5924af6c\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:17.931024 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:17.930951    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d9b5f5cd-88jk9" event={"ID":"e6b11b6f-c402-4307-919c-add8df8f4538","Type":"ContainerStarted","Data":"4649ad967ec38f9f2d6790a85bb95c1a183c81ef7af13e251e134a77ff00da83"}
Jul 15 19:30:18.111576 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:18.111485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf"
Jul 15 19:30:18.286813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:18.286740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cv4rf"]
Jul 15 19:30:18.292849 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:18.292818    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ec833a_e136_4858_b688_9eed5924af6c.slice/crio-3358882a1ee8103c4456a25c3bd36317f0e073786e56c9ca009722d4529329ac WatchSource:0}: Error finding container 3358882a1ee8103c4456a25c3bd36317f0e073786e56c9ca009722d4529329ac: Status 404 returned error can't find the container with id 3358882a1ee8103c4456a25c3bd36317f0e073786e56c9ca009722d4529329ac
Jul 15 19:30:18.514975 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:18.514933    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:18.568986 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:18.568942    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:18.938278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:18.938180    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf" event={"ID":"d2ec833a-e136-4858-b688-9eed5924af6c","Type":"ContainerStarted","Data":"3358882a1ee8103c4456a25c3bd36317f0e073786e56c9ca009722d4529329ac"}
Jul 15 19:30:19.326194 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:19.326128    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:19.381171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:19.381131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:22.123008 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:22.122963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:22.175889 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:22.175856    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:22.897729 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:22.897699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rktvs"
Jul 15 19:30:29.158075 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:29.158039    2564 patch_prober.go:28] interesting pod/image-registry-56699b974f-hmvkq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 15 19:30:29.158483 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:29.158098    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 19:30:30.085240 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.085194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56699b974f-hmvkq"]
Jul 15 19:30:30.090857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.090773    2564 patch_prober.go:28] interesting pod/image-registry-56699b974f-hmvkq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 15 19:30:30.090857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.090835    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 19:30:30.161397 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.161328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:30.259968 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.259890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:30.533153 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.533116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:30.996420 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.996377    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b98778d74-pfhxh" event={"ID":"ce7c9a99-ca74-4c17-ab04-4d6d61d18384","Type":"ContainerStarted","Data":"536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425"}
Jul 15 19:30:30.998126 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.998094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-d7hbg" event={"ID":"b05d4d84-59fb-4855-83d2-5f285ffcea40","Type":"ContainerStarted","Data":"db588a5ee44f87206c371ea27a8e15efa58ae56b5f042219e83e89937ebd2277"}
Jul 15 19:30:30.998297 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:30.998215    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:31.000727 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.000689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6brrv" event={"ID":"398023a0-b103-4631-9812-b1785776bfe6","Type":"ContainerStarted","Data":"8864f66b1e6e3ac0aea3b11d2d2c5a51ceee7de199c0e2d0bf675b4d9b67d391"}
Jul 15 19:30:31.002402 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.002374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d9b5f5cd-88jk9" event={"ID":"e6b11b6f-c402-4307-919c-add8df8f4538","Type":"ContainerStarted","Data":"2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009"}
Jul 15 19:30:31.004112 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.004091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf" event={"ID":"d2ec833a-e136-4858-b688-9eed5924af6c","Type":"ContainerStarted","Data":"07de10228e816ecda0f3a80fc68e239958f4f92530492fc5c26f11a6419e6ee7"}
Jul 15 19:30:31.004189 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.004128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf" event={"ID":"d2ec833a-e136-4858-b688-9eed5924af6c","Type":"ContainerStarted","Data":"015fae0304de5df2c5957132b4fc2cc3187fcfe5736112f42ac302fc5a7a924d"}
Jul 15 19:30:31.013083 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.013033    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b98778d74-pfhxh" podStartSLOduration=1.892476295 podStartE2EDuration="16.013016248s" totalImagesPullingTime="14.120539953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:31.011432476 +0000 UTC m=+121.089225278" watchObservedRunningTime="2026-07-15 19:30:31.013016248 +0000 UTC m=+121.090809048"
Jul 15 19:30:31.013929 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.013906    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-d7hbg"
Jul 15 19:30:31.030512 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.030460    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-cv4rf" podStartSLOduration=2.143857125 podStartE2EDuration="14.030447445s" totalImagesPullingTime="11.88659032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:31.029477979 +0000 UTC m=+121.107270780" watchObservedRunningTime="2026-07-15 19:30:31.030447445 +0000 UTC m=+121.108240228"
Jul 15 19:30:31.049097 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.049033    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54d9b5f5cd-88jk9" podStartSLOduration=1.8275898179999999 podStartE2EDuration="14.049013686s" totalImagesPullingTime="12.221423868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:31.047812755 +0000 UTC m=+121.125605557" watchObservedRunningTime="2026-07-15 19:30:31.049013686 +0000 UTC m=+121.126806491"
Jul 15 19:30:31.062754 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:31.062698    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-d7hbg" podStartSLOduration=1.472491167 podStartE2EDuration="23.062678225s" totalImagesPullingTime="21.590187058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:31.062212171 +0000 UTC m=+121.140004972" watchObservedRunningTime="2026-07-15 19:30:31.062678225 +0000 UTC m=+121.140471027"
Jul 15 19:30:32.008929 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:32.008555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:32.109800 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:32.109709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:32.874528 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:32.874475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:33.093504 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.093469    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:33.104851 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.104823    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"]
Jul 15 19:30:33.148433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.148353    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"]
Jul 15 19:30:33.148433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.148414    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vhddg"]
Jul 15 19:30:33.149367 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.149337    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.151792 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.151771    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-55xsh\""
Jul 15 19:30:33.152074 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.152030    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 15 19:30:33.152258 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.152214    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 15 19:30:33.170615 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.170582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:33.179618 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.179588    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.183335 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.182198    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 19:30:33.183335 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.182482    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 19:30:33.183335 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.182696    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x2sbw\""
Jul 15 19:30:33.183335 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.182896    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-textfile\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-sys\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-tls\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/036e2a6c-b057-44ce-9778-0540d4f5d4a2-metrics-client-ca\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-wtmp\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263499    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrz8\" (UniqueName: \"kubernetes.io/projected/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-kube-api-access-8jrz8\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263558    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rklj7\" (UniqueName: \"kubernetes.io/projected/036e2a6c-b057-44ce-9778-0540d4f5d4a2-kube-api-access-rklj7\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263597    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-root\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.263813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.263641    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.306016 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.305923    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-sys\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-tls\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/036e2a6c-b057-44ce-9778-0540d4f5d4a2-metrics-client-ca\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-wtmp\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrz8\" (UniqueName: \"kubernetes.io/projected/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-kube-api-access-8jrz8\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364690    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rklj7\" (UniqueName: \"kubernetes.io/projected/036e2a6c-b057-44ce-9778-0540d4f5d4a2-kube-api-access-rklj7\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-root\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.364809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-textfile\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.365165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-textfile\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.365259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-sys\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.366345    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/036e2a6c-b057-44ce-9778-0540d4f5d4a2-metrics-client-ca\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367925 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.366450    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.367925 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:30:33.366536    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 15 19:30:33.367925 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:30:33.366590    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-tls podName:a7fbabbe-3115-4f5d-b281-66a1e05c5a27 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:33.866576587 +0000 UTC m=+123.944369365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-s9tvz" (UID: "a7fbabbe-3115-4f5d-b281-66a1e05c5a27") : secret "openshift-state-metrics-tls" not found
Jul 15 19:30:33.367925 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.366794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-root\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367925 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.366879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-wtmp\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.367925 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.367107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.370378 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.370199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.371343 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.371306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-tls\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.373263 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.373205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/036e2a6c-b057-44ce-9778-0540d4f5d4a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.383552 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.383041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrz8\" (UniqueName: \"kubernetes.io/projected/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-kube-api-access-8jrz8\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.385950 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.385917    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rklj7\" (UniqueName: \"kubernetes.io/projected/036e2a6c-b057-44ce-9778-0540d4f5d4a2-kube-api-access-rklj7\") pod \"node-exporter-vhddg\" (UID: \"036e2a6c-b057-44ce-9778-0540d4f5d4a2\") " pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.494662 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.494610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vhddg"
Jul 15 19:30:33.869413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.869266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:33.872574 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:33.872546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7fbabbe-3115-4f5d-b281-66a1e05c5a27-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-s9tvz\" (UID: \"a7fbabbe-3115-4f5d-b281-66a1e05c5a27\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:34.063736 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:34.063683    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"
Jul 15 19:30:34.087817 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:34.087786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:34.179314 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:34.179267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:34.298921 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:34.298841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:34.498172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:34.498056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:35.096351 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.096322    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"]
Jul 15 19:30:35.148627 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.148590    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"]
Jul 15 19:30:35.148817 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.148726    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.152327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.151652    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 19:30:35.152327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.151686    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 19:30:35.152327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.151899    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 19:30:35.152327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.151921    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 19:30:35.152327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.152094    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-324lna5es5h1r\""
Jul 15 19:30:35.152327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.152182    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9fzwk\""
Jul 15 19:30:35.153188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.153167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 19:30:35.286321 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286283    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-grpc-tls\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286399    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd6sm\" (UniqueName: \"kubernetes.io/projected/d795ccaf-767d-4335-909b-8fcbb1ebbea8-kube-api-access-gd6sm\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d795ccaf-767d-4335-909b-8fcbb1ebbea8-metrics-client-ca\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-tls\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.286791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.286637    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.387648 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.387561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd6sm\" (UniqueName: \"kubernetes.io/projected/d795ccaf-767d-4335-909b-8fcbb1ebbea8-kube-api-access-gd6sm\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.387648 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.387615    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.387876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.387772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d795ccaf-767d-4335-909b-8fcbb1ebbea8-metrics-client-ca\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.387876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.387822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-tls\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.388016 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.387901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.388086 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.388017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.389371 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.388579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.389371 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.388625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-grpc-tls\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.389371 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.389156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d795ccaf-767d-4335-909b-8fcbb1ebbea8-metrics-client-ca\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.392672 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.392649    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.393721 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.393675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-tls\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.394256 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.394213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.396386 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.396341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.398258 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.398206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.399481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.399456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd6sm\" (UniqueName: \"kubernetes.io/projected/d795ccaf-767d-4335-909b-8fcbb1ebbea8-kube-api-access-gd6sm\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.408294 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.408244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d795ccaf-767d-4335-909b-8fcbb1ebbea8-secret-grpc-tls\") pod \"thanos-querier-cc6f8fcb4-f5vsz\" (UID: \"d795ccaf-767d-4335-909b-8fcbb1ebbea8\") " pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.464465 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.464174    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:35.813649 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:35.813616    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036e2a6c_b057_44ce_9778_0540d4f5d4a2.slice/crio-3c8c5c9e8ac570baffc4e885cc81321258507db29fb5b08e64610ced4ec24916 WatchSource:0}: Error finding container 3c8c5c9e8ac570baffc4e885cc81321258507db29fb5b08e64610ced4ec24916: Status 404 returned error can't find the container with id 3c8c5c9e8ac570baffc4e885cc81321258507db29fb5b08e64610ced4ec24916
Jul 15 19:30:35.881926 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.880723    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:35.881926 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.880838    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:35.894304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.894211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:35.979665 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:35.979554    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz"]
Jul 15 19:30:35.985892 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:35.985839    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fbabbe_3115_4f5d_b281_66a1e05c5a27.slice/crio-1c91666ed7a8d9b858f6745cef91a59f73d3fa8cfc0cece30ae109225aab269c WatchSource:0}: Error finding container 1c91666ed7a8d9b858f6745cef91a59f73d3fa8cfc0cece30ae109225aab269c: Status 404 returned error can't find the container with id 1c91666ed7a8d9b858f6745cef91a59f73d3fa8cfc0cece30ae109225aab269c
Jul 15 19:30:36.013691 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.013579    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"]
Jul 15 19:30:36.017380 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:36.017352    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd795ccaf_767d_4335_909b_8fcbb1ebbea8.slice/crio-1570ba04b5b745dee853560fb69a9e438026ea4cc42edda094052af53dd5e9e8 WatchSource:0}: Error finding container 1570ba04b5b745dee853560fb69a9e438026ea4cc42edda094052af53dd5e9e8: Status 404 returned error can't find the container with id 1570ba04b5b745dee853560fb69a9e438026ea4cc42edda094052af53dd5e9e8
Jul 15 19:30:36.022177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.022153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:36.027052 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.026958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"1570ba04b5b745dee853560fb69a9e438026ea4cc42edda094052af53dd5e9e8"}
Jul 15 19:30:36.028348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.028321    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz" event={"ID":"a7fbabbe-3115-4f5d-b281-66a1e05c5a27","Type":"ContainerStarted","Data":"1c91666ed7a8d9b858f6745cef91a59f73d3fa8cfc0cece30ae109225aab269c"}
Jul 15 19:30:36.030761 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.030738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vhddg" event={"ID":"036e2a6c-b057-44ce-9778-0540d4f5d4a2","Type":"ContainerStarted","Data":"3c8c5c9e8ac570baffc4e885cc81321258507db29fb5b08e64610ced4ec24916"}
Jul 15 19:30:36.035895 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.035877    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:30:36.107798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.107767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:36.210914 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.210872    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:36.364640 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.364617    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:36.567924 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:36.567878    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:37.040188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.040131    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6brrv" event={"ID":"398023a0-b103-4631-9812-b1785776bfe6","Type":"ContainerStarted","Data":"81e398c7a62bf47472fd0b786bdeec895add1b3a9852f81f07392a7a449117ee"}
Jul 15 19:30:37.044740 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.044631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz" event={"ID":"a7fbabbe-3115-4f5d-b281-66a1e05c5a27","Type":"ContainerStarted","Data":"ece14973f6d0a23c264e4b681bc8e1994c5eb59afa42b0e265a1b07636dd8c45"}
Jul 15 19:30:37.044740 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.044688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz" event={"ID":"a7fbabbe-3115-4f5d-b281-66a1e05c5a27","Type":"ContainerStarted","Data":"858188af5a17a5038713a9967e63d4a33236eb6bd0b292e9c184f703bf0aeb9e"}
Jul 15 19:30:37.421973 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.420626    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"]
Jul 15 19:30:37.442296 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.441015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"]
Jul 15 19:30:37.442296 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.441162    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.444252 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.443824    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 19:30:37.444252 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.443969    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9s9h9\""
Jul 15 19:30:37.447127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.446375    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fk3hjak05tra7\""
Jul 15 19:30:37.447127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.446595    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 19:30:37.447127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.446777    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 19:30:37.447127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.446848    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:37.447127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.446887    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:37.447127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.446958    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 19:30:37.456122 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.456073    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4313cff1-9090-46cc-8186-0bed2a37c32e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514494    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-client-ca-bundle\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-secret-metrics-server-tls\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-secret-metrics-server-client-certs\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4313cff1-9090-46cc-8186-0bed2a37c32e-audit-log\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qw47\" (UniqueName: \"kubernetes.io/projected/4313cff1-9090-46cc-8186-0bed2a37c32e-kube-api-access-6qw47\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.514826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.514682    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4313cff1-9090-46cc-8186-0bed2a37c32e-metrics-server-audit-profiles\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616356 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4313cff1-9090-46cc-8186-0bed2a37c32e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616515 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616373    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-client-ca-bundle\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616515 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-secret-metrics-server-tls\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616515 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-secret-metrics-server-client-certs\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616657 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4313cff1-9090-46cc-8186-0bed2a37c32e-audit-log\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616657 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qw47\" (UniqueName: \"kubernetes.io/projected/4313cff1-9090-46cc-8186-0bed2a37c32e-kube-api-access-6qw47\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.616736 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.616676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4313cff1-9090-46cc-8186-0bed2a37c32e-metrics-server-audit-profiles\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.617852 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.617824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4313cff1-9090-46cc-8186-0bed2a37c32e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.622130 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.618490    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4313cff1-9090-46cc-8186-0bed2a37c32e-audit-log\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.622130 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.621541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4313cff1-9090-46cc-8186-0bed2a37c32e-metrics-server-audit-profiles\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.622130 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.621548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-secret-metrics-server-tls\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.622130 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.622092    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-secret-metrics-server-client-certs\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.623057 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.623014    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4313cff1-9090-46cc-8186-0bed2a37c32e-client-ca-bundle\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.627659 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.627633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qw47\" (UniqueName: \"kubernetes.io/projected/4313cff1-9090-46cc-8186-0bed2a37c32e-kube-api-access-6qw47\") pod \"metrics-server-8545cf9cdb-tvdjz\" (UID: \"4313cff1-9090-46cc-8186-0bed2a37c32e\") " pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.759076 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.758999    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:37.938197 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.936526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:37.948702 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.943967    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54d9b5f5cd-88jk9"]
Jul 15 19:30:37.969138 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.969110    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797894dcc7-q9bnn"]
Jul 15 19:30:37.985659 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.984967    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"]
Jul 15 19:30:37.986099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.985363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:37.986779 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:37.985861    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797894dcc7-q9bnn"]
Jul 15 19:30:38.010186 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.009821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:38.056282 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.055712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vhddg" event={"ID":"036e2a6c-b057-44ce-9778-0540d4f5d4a2","Type":"ContainerStarted","Data":"1e3888d412d02b6ebf347b238f5325902b1e690adb82181fa3016d28eb5b9bec"}
Jul 15 19:30:38.065541 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.065473    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123496    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-trusted-ca-bundle\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123548    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-oauth-serving-cert\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-oauth-config\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123658    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-config\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-service-ca\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-serving-cert\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.126774 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.123794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44tq4\" (UniqueName: \"kubernetes.io/projected/6a87c8c0-4f74-4d6d-a270-3c8037aef389-kube-api-access-44tq4\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.224694 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-trusted-ca-bundle\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.224883 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-oauth-serving-cert\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.224883 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-oauth-config\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.224883 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-config\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.225059 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-service-ca\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.225059 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-serving-cert\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.225059 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.224954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44tq4\" (UniqueName: \"kubernetes.io/projected/6a87c8c0-4f74-4d6d-a270-3c8037aef389-kube-api-access-44tq4\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.226889 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.226864    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-trusted-ca-bundle\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.227477 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.227446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-oauth-serving-cert\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.228652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.227980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-config\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.228652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.228564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-service-ca\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.229680 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.229643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-serving-cert\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.230008 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.229965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-oauth-config\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.234081 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.234038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44tq4\" (UniqueName: \"kubernetes.io/projected/6a87c8c0-4f74-4d6d-a270-3c8037aef389-kube-api-access-44tq4\") pod \"console-797894dcc7-q9bnn\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") " pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.301780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.301598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:38.424204 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:38.424121    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:39.032705 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:39.032669    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4313cff1_9090_46cc_8186_0bed2a37c32e.slice/crio-fbfe4a5eabd467ce9fe5ff1887edf79006cd40baa20be5467ef96def244943a4 WatchSource:0}: Error finding container fbfe4a5eabd467ce9fe5ff1887edf79006cd40baa20be5467ef96def244943a4: Status 404 returned error can't find the container with id fbfe4a5eabd467ce9fe5ff1887edf79006cd40baa20be5467ef96def244943a4
Jul 15 19:30:39.073584 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:39.071007    2564 generic.go:356] "Generic (PLEG): container finished" podID="036e2a6c-b057-44ce-9778-0540d4f5d4a2" containerID="1e3888d412d02b6ebf347b238f5325902b1e690adb82181fa3016d28eb5b9bec" exitCode=0
Jul 15 19:30:39.073584 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:39.071123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vhddg" event={"ID":"036e2a6c-b057-44ce-9778-0540d4f5d4a2","Type":"ContainerDied","Data":"1e3888d412d02b6ebf347b238f5325902b1e690adb82181fa3016d28eb5b9bec"}
Jul 15 19:30:39.077439 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:39.075735    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz" event={"ID":"4313cff1-9090-46cc-8186-0bed2a37c32e","Type":"ContainerStarted","Data":"fbfe4a5eabd467ce9fe5ff1887edf79006cd40baa20be5467ef96def244943a4"}
Jul 15 19:30:39.747122 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:39.747083    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:39.844209 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:39.844173    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:39.946039 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:39.946010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:40.092537 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:40.092512    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:40.250426 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:40.249848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:30:40.253166 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:40.253124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88f9a415-fea9-462f-83bf-e457ab604332-metrics-certs\") pod \"network-metrics-daemon-9pclt\" (UID: \"88f9a415-fea9-462f-83bf-e457ab604332\") " pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:30:40.309389 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:40.309356    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t5v82\""
Jul 15 19:30:40.317307 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:40.317261    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9pclt"
Jul 15 19:30:41.108727 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:41.108692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:41.769958 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:41.769920    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:42.246079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:42.246057    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797894dcc7-q9bnn"]
Jul 15 19:30:42.250534 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:42.250418    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a87c8c0_4f74_4d6d_a270_3c8037aef389.slice/crio-df3a7c7b8543d3588bf7fb38d654dbd1f144fccee86906c4ecf9d9517fbab201 WatchSource:0}: Error finding container df3a7c7b8543d3588bf7fb38d654dbd1f144fccee86906c4ecf9d9517fbab201: Status 404 returned error can't find the container with id df3a7c7b8543d3588bf7fb38d654dbd1f144fccee86906c4ecf9d9517fbab201
Jul 15 19:30:42.265964 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:42.265938    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9pclt"]
Jul 15 19:30:42.268811 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:30:42.268790    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f9a415_fea9_462f_83bf_e457ab604332.slice/crio-f09ef2ac5804402393cd696ff46f5a71e898317b1e6c7c24fb15080b25eb4a7b WatchSource:0}: Error finding container f09ef2ac5804402393cd696ff46f5a71e898317b1e6c7c24fb15080b25eb4a7b: Status 404 returned error can't find the container with id f09ef2ac5804402393cd696ff46f5a71e898317b1e6c7c24fb15080b25eb4a7b
Jul 15 19:30:42.284312 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:42.284285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:42.386736 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:42.386706    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:43.100832 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.100801    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9pclt" event={"ID":"88f9a415-fea9-462f-83bf-e457ab604332","Type":"ContainerStarted","Data":"f09ef2ac5804402393cd696ff46f5a71e898317b1e6c7c24fb15080b25eb4a7b"}
Jul 15 19:30:43.102968 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.102938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"34fe8c7f82de8555b8c691bc2d2680fcd45ae3b84d35e4e4092c11dd385bf706"}
Jul 15 19:30:43.103076 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.103006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"0d1365489eb870a884dffca96451671b759db115d99d0a7bb59977f09ad1b12a"}
Jul 15 19:30:43.107724 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.107695    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz" event={"ID":"a7fbabbe-3115-4f5d-b281-66a1e05c5a27","Type":"ContainerStarted","Data":"aa8c8a48e13bc7f463e2080bf1a8c9cba08ec03e1f790c6941170401046b0020"}
Jul 15 19:30:43.109988 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.109966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vhddg" event={"ID":"036e2a6c-b057-44ce-9778-0540d4f5d4a2","Type":"ContainerStarted","Data":"c24c6b2dd5e263b357244e3b1e44eb70d66d7da5a3f9530e52d168c44f253da5"}
Jul 15 19:30:43.110106 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.110014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vhddg" event={"ID":"036e2a6c-b057-44ce-9778-0540d4f5d4a2","Type":"ContainerStarted","Data":"4d3b7aaa0737b805e23054b2664ac925e02fe78f72421b7ffc886c151c7b3133"}
Jul 15 19:30:43.111566 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.111545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797894dcc7-q9bnn" event={"ID":"6a87c8c0-4f74-4d6d-a270-3c8037aef389","Type":"ContainerStarted","Data":"c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e"}
Jul 15 19:30:43.111672 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.111583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797894dcc7-q9bnn" event={"ID":"6a87c8c0-4f74-4d6d-a270-3c8037aef389","Type":"ContainerStarted","Data":"df3a7c7b8543d3588bf7fb38d654dbd1f144fccee86906c4ecf9d9517fbab201"}
Jul 15 19:30:43.126849 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.126807    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-s9tvz" podStartSLOduration=4.422305818 podStartE2EDuration="10.126795267s" totalImagesPullingTime="5.704489449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:43.126607794 +0000 UTC m=+133.204400594" watchObservedRunningTime="2026-07-15 19:30:43.126795267 +0000 UTC m=+133.204588068"
Jul 15 19:30:43.150204 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.150160    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797894dcc7-q9bnn" podStartSLOduration=6.150145548 podStartE2EDuration="6.150145548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:43.149613403 +0000 UTC m=+133.227406205" watchObservedRunningTime="2026-07-15 19:30:43.150145548 +0000 UTC m=+133.227938327"
Jul 15 19:30:43.170033 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:43.169981    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vhddg" podStartSLOduration=8.104775531 podStartE2EDuration="10.169970818s" totalImagesPullingTime="1.951946046s" totalInitContainerRuntime="113.249241ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:43.168682089 +0000 UTC m=+133.246474892" watchObservedRunningTime="2026-07-15 19:30:43.169970818 +0000 UTC m=+133.247763618"
Jul 15 19:30:44.120569 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:44.120530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6brrv" event={"ID":"398023a0-b103-4631-9812-b1785776bfe6","Type":"ContainerStarted","Data":"1a2e28e75278487f26adba60826bf85fa5e8046a668dac2d5cd2a2eb1d980be9"}
Jul 15 19:30:44.123107 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:44.123078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"0d5424f64b2b624e2fcf497ca7d784d2cb199afea176c5ead09927c451ecfca2"}
Jul 15 19:30:44.127290 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:44.127256    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz" event={"ID":"4313cff1-9090-46cc-8186-0bed2a37c32e","Type":"ContainerStarted","Data":"82325c4c4d92c3f5351da53e2aaf1c1455323c12865e5b35c8195583817817a6"}
Jul 15 19:30:44.140037 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:44.139982    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6brrv" podStartSLOduration=34.016417415 podStartE2EDuration="1m1.139961197s" totalImagesPullingTime="27.123543782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:29:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:44.138539121 +0000 UTC m=+134.216331926" watchObservedRunningTime="2026-07-15 19:30:44.139961197 +0000 UTC m=+134.217754001"
Jul 15 19:30:44.156384 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:44.156325    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz" podStartSLOduration=2.881225697 podStartE2EDuration="7.156304375s" totalImagesPullingTime="4.275078678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:44.154274596 +0000 UTC m=+134.232067397" watchObservedRunningTime="2026-07-15 19:30:44.156304375 +0000 UTC m=+134.234097177"
Jul 15 19:30:45.134481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:45.134389    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9pclt" event={"ID":"88f9a415-fea9-462f-83bf-e457ab604332","Type":"ContainerStarted","Data":"2bb0b7d734a1dede67ee3da8427140bdb6cf1a48c09c8952458f2d121738dde6"}
Jul 15 19:30:45.134481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:45.134469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9pclt" event={"ID":"88f9a415-fea9-462f-83bf-e457ab604332","Type":"ContainerStarted","Data":"ff341888eef3103ee5e07bdb1c84513ad34c4026c9288541ee529966805d5913"}
Jul 15 19:30:45.150122 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:45.150077    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9pclt" podStartSLOduration=132.968631053 podStartE2EDuration="2m15.150062213s" totalImagesPullingTime="2.18143116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:45.149556383 +0000 UTC m=+135.227349184" watchObservedRunningTime="2026-07-15 19:30:45.150062213 +0000 UTC m=+135.227855014"
Jul 15 19:30:45.239487 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:45.239456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:45.340162 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:45.340120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:46.144628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:46.144584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"5927a268836b7cd21d9efea44e6a5ac621ed01ee8bc5d3bb53b8ada3a8dae5e2"}
Jul 15 19:30:46.145128 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:46.144664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"ea8aa13e41a10ab4fa63b67f88928aecd803f1305d232644d5f9d6f164ce8025"}
Jul 15 19:30:46.145128 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:46.144693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" event={"ID":"d795ccaf-767d-4335-909b-8fcbb1ebbea8","Type":"ContainerStarted","Data":"67b68dd8cd5694bbee8915f84f9ec314627ed95897efdbff57b1ed3e2bba4a28"}
Jul 15 19:30:46.145128 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:46.144963    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:46.166162 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:46.166100    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz" podStartSLOduration=3.218533985 podStartE2EDuration="11.166084563s" totalImagesPullingTime="7.947550578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:46.164473298 +0000 UTC m=+136.242266096" watchObservedRunningTime="2026-07-15 19:30:46.166084563 +0000 UTC m=+136.243877363"
Jul 15 19:30:48.302377 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:48.302331    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:48.302758 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:48.302411    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:48.307171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:48.307147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:48.856976 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:48.856927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:48.957873 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:48.957837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.159942 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:49.159863    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:30:49.218475 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:49.218425    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b98778d74-pfhxh"]
Jul 15 19:30:49.342183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:49.342147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.442237 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:49.442136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.497034 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:49.496991    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:51.285172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:51.285133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:51.389829 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:51.389795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:51.485539 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:51.485495    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:51.599999 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:51.599907    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:52.154243 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:52.154204    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-cc6f8fcb4-f5vsz"
Jul 15 19:30:52.379373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:52.379337    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:52.480547 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:52.480514    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:53.346607 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:53.346573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:53.447112 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:53.447072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:55.108016 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.107980    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" containerName="registry" containerID="cri-o://9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a" gracePeriod=30
Jul 15 19:30:55.382297 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.382215    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:55.393993 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.393970    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6slv\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-kube-api-access-d6slv\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394026    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-image-registry-private-configuration\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46844f3e-e443-47a1-a487-82d143fed4c9-ca-trust-extracted\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394085    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-installation-pull-secrets\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394119    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-registry-certificates\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394137    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-trusted-ca\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394411 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394411 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-bound-sa-token\") pod \"46844f3e-e443-47a1-a487-82d143fed4c9\" (UID: \"46844f3e-e443-47a1-a487-82d143fed4c9\") "
Jul 15 19:30:55.394682 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.394649    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:30:55.395415 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.395380    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:30:55.396958 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.396911    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:30:55.397181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.397104    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:30:55.397181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.397122    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:30:55.397340 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.397214    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-kube-api-access-d6slv" (OuterVolumeSpecName: "kube-api-access-d6slv") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "kube-api-access-d6slv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:30:55.397395 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.397321    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:30:55.404392 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.404361    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46844f3e-e443-47a1-a487-82d143fed4c9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "46844f3e-e443-47a1-a487-82d143fed4c9" (UID: "46844f3e-e443-47a1-a487-82d143fed4c9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:30:55.495854 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495817    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-bound-sa-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.495854 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495855    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6slv\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-kube-api-access-d6slv\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.496064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495874    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-image-registry-private-configuration\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.496064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495890    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46844f3e-e443-47a1-a487-82d143fed4c9-ca-trust-extracted\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.496064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495903    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46844f3e-e443-47a1-a487-82d143fed4c9-installation-pull-secrets\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.496064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495917    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-registry-certificates\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.496064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495929    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46844f3e-e443-47a1-a487-82d143fed4c9-trusted-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:55.496064 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:55.495942    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46844f3e-e443-47a1-a487-82d143fed4c9-registry-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:30:56.183306 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.183270    2564 generic.go:356] "Generic (PLEG): container finished" podID="46844f3e-e443-47a1-a487-82d143fed4c9" containerID="9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a" exitCode=0
Jul 15 19:30:56.183932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.183315    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56699b974f-hmvkq"
Jul 15 19:30:56.183932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.183358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" event={"ID":"46844f3e-e443-47a1-a487-82d143fed4c9","Type":"ContainerDied","Data":"9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a"}
Jul 15 19:30:56.183932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.183417    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56699b974f-hmvkq" event={"ID":"46844f3e-e443-47a1-a487-82d143fed4c9","Type":"ContainerDied","Data":"3e4be9fd2d49a5c041db657a82f3a10bc2b26fc7c2b65177c11ea42dd2b23c82"}
Jul 15 19:30:56.183932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.183451    2564 scope.go:117] "RemoveContainer" containerID="9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a"
Jul 15 19:30:56.192678 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.192658    2564 scope.go:117] "RemoveContainer" containerID="9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a"
Jul 15 19:30:56.192937 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:30:56.192916    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a\": container with ID starting with 9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a not found: ID does not exist" containerID="9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a"
Jul 15 19:30:56.192984 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.192950    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a"} err="failed to get container status \"9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a\": rpc error: code = NotFound desc = could not find container \"9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a\": container with ID starting with 9021a7ea56c9d55a8ec82f61351921865647d31d8584d3cdd1c4fd9893379e2a not found: ID does not exist"
Jul 15 19:30:56.202576 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.202548    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56699b974f-hmvkq"]
Jul 15 19:30:56.206068 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.206046    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56699b974f-hmvkq"]
Jul 15 19:30:56.491749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:56.491716    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" path="/var/lib/kubelet/pods/46844f3e-e443-47a1-a487-82d143fed4c9/volumes"
Jul 15 19:30:57.760079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:57.759991    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:57.760079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:57.760064    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:30:58.194247 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:58.194148    2564 generic.go:356] "Generic (PLEG): container finished" podID="22a24f49-4050-4490-aaaa-0a317701ded1" containerID="e1d2984375cb197be2c852a6e6aad319dadf629f4b3cf1d69321ed50ffe60070" exitCode=0
Jul 15 19:30:58.194247 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:58.194202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps" event={"ID":"22a24f49-4050-4490-aaaa-0a317701ded1","Type":"ContainerDied","Data":"e1d2984375cb197be2c852a6e6aad319dadf629f4b3cf1d69321ed50ffe60070"}
Jul 15 19:30:58.194557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:58.194544    2564 scope.go:117] "RemoveContainer" containerID="e1d2984375cb197be2c852a6e6aad319dadf629f4b3cf1d69321ed50ffe60070"
Jul 15 19:30:58.950418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:58.950383    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9ccbbdb-bqbnh_299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2/router/0.log"
Jul 15 19:30:58.968041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:58.968005    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7pcjt_e628e49a-bcdb-46c6-91d2-e6942e8b1eb5/serve-healthcheck-canary/0.log"
Jul 15 19:30:59.203206 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:30:59.203174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fp5ps" event={"ID":"22a24f49-4050-4490-aaaa-0a317701ded1","Type":"ContainerStarted","Data":"b41b01e9e9491288a51030913fa03f7b35639c4d3965d0eda4af9643405c0030"}
Jul 15 19:31:01.971656 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:01.971621    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:02.084966 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:02.084930    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:02.138996 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:02.138964    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:04.118897 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.118847    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54d9b5f5cd-88jk9" podUID="e6b11b6f-c402-4307-919c-add8df8f4538" containerName="console" containerID="cri-o://2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009" gracePeriod=15
Jul 15 19:31:04.422433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.422399    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:31:04.473406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473372    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-oauth-config\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473410    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-oauth-serving-cert\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473444    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bph28\" (UniqueName: \"kubernetes.io/projected/e6b11b6f-c402-4307-919c-add8df8f4538-kube-api-access-bph28\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473478    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-console-config\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473508    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-service-ca\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473547    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-trusted-ca-bundle\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473628 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473608    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-serving-cert\") pod \"e6b11b6f-c402-4307-919c-add8df8f4538\" (UID: \"e6b11b6f-c402-4307-919c-add8df8f4538\") "
Jul 15 19:31:04.473933 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473904    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:04.474014 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473952    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-service-ca" (OuterVolumeSpecName: "service-ca") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:04.474014 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473975    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-console-config" (OuterVolumeSpecName: "console-config") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:04.474014 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.473984    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:04.475873 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.475848    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:31:04.475969 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.475872    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b11b6f-c402-4307-919c-add8df8f4538-kube-api-access-bph28" (OuterVolumeSpecName: "kube-api-access-bph28") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "kube-api-access-bph28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:31:04.475969 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.475869    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e6b11b6f-c402-4307-919c-add8df8f4538" (UID: "e6b11b6f-c402-4307-919c-add8df8f4538"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:31:04.575012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.574970    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bph28\" (UniqueName: \"kubernetes.io/projected/e6b11b6f-c402-4307-919c-add8df8f4538-kube-api-access-bph28\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:04.575012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.575005    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:04.575012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.575014    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:04.575012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.575024    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:04.575305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.575033    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:04.575305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.575041    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6b11b6f-c402-4307-919c-add8df8f4538-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:04.575305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:04.575049    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6b11b6f-c402-4307-919c-add8df8f4538-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:05.228569 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.228530    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6b11b6f-c402-4307-919c-add8df8f4538" containerID="2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009" exitCode=0
Jul 15 19:31:05.228937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.228593    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d9b5f5cd-88jk9"
Jul 15 19:31:05.228937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.228620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d9b5f5cd-88jk9" event={"ID":"e6b11b6f-c402-4307-919c-add8df8f4538","Type":"ContainerDied","Data":"2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009"}
Jul 15 19:31:05.228937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.228676    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d9b5f5cd-88jk9" event={"ID":"e6b11b6f-c402-4307-919c-add8df8f4538","Type":"ContainerDied","Data":"4649ad967ec38f9f2d6790a85bb95c1a183c81ef7af13e251e134a77ff00da83"}
Jul 15 19:31:05.228937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.228698    2564 scope.go:117] "RemoveContainer" containerID="2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009"
Jul 15 19:31:05.237455 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.237436    2564 scope.go:117] "RemoveContainer" containerID="2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009"
Jul 15 19:31:05.237766 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:31:05.237741    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009\": container with ID starting with 2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009 not found: ID does not exist" containerID="2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009"
Jul 15 19:31:05.237820 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.237776    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009"} err="failed to get container status \"2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009\": rpc error: code = NotFound desc = could not find container \"2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009\": container with ID starting with 2c82372897604ce9fb18c7d53df541b608d8a183e32daa8307f0407bbba9d009 not found: ID does not exist"
Jul 15 19:31:05.242210 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.242188    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54d9b5f5cd-88jk9"]
Jul 15 19:31:05.244471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:05.244451    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54d9b5f5cd-88jk9"]
Jul 15 19:31:06.492501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:06.492461    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b11b6f-c402-4307-919c-add8df8f4538" path="/var/lib/kubelet/pods/e6b11b6f-c402-4307-919c-add8df8f4538/volumes"
Jul 15 19:31:08.340241 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:08.340195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:08.393125 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:08.393085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:14.257079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:14.257004    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b98778d74-pfhxh" podUID="ce7c9a99-ca74-4c17-ab04-4d6d61d18384" containerName="console" containerID="cri-o://536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425" gracePeriod=15
Jul 15 19:31:15.032451 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.032421    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:31:15.172154 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172070    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-service-ca\") pod \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") "
Jul 15 19:31:15.172154 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172126    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-oauth-config\") pod \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") "
Jul 15 19:31:15.172154 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172154    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-config\") pod \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") "
Jul 15 19:31:15.172460 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172172    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-oauth-serving-cert\") pod \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") "
Jul 15 19:31:15.172460 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd7hr\" (UniqueName: \"kubernetes.io/projected/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-kube-api-access-cd7hr\") pod \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") "
Jul 15 19:31:15.172460 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-serving-cert\") pod \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\" (UID: \"ce7c9a99-ca74-4c17-ab04-4d6d61d18384\") "
Jul 15 19:31:15.172629 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172600    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-config" (OuterVolumeSpecName: "console-config") pod "ce7c9a99-ca74-4c17-ab04-4d6d61d18384" (UID: "ce7c9a99-ca74-4c17-ab04-4d6d61d18384"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:15.172691 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172603    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-service-ca" (OuterVolumeSpecName: "service-ca") pod "ce7c9a99-ca74-4c17-ab04-4d6d61d18384" (UID: "ce7c9a99-ca74-4c17-ab04-4d6d61d18384"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:15.172691 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.172665    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ce7c9a99-ca74-4c17-ab04-4d6d61d18384" (UID: "ce7c9a99-ca74-4c17-ab04-4d6d61d18384"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:31:15.174684 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.174658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-kube-api-access-cd7hr" (OuterVolumeSpecName: "kube-api-access-cd7hr") pod "ce7c9a99-ca74-4c17-ab04-4d6d61d18384" (UID: "ce7c9a99-ca74-4c17-ab04-4d6d61d18384"). InnerVolumeSpecName "kube-api-access-cd7hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:31:15.174684 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.174658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ce7c9a99-ca74-4c17-ab04-4d6d61d18384" (UID: "ce7c9a99-ca74-4c17-ab04-4d6d61d18384"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:31:15.174805 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.174736    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ce7c9a99-ca74-4c17-ab04-4d6d61d18384" (UID: "ce7c9a99-ca74-4c17-ab04-4d6d61d18384"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:31:15.272477 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272442    2564 generic.go:356] "Generic (PLEG): container finished" podID="ce7c9a99-ca74-4c17-ab04-4d6d61d18384" containerID="536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425" exitCode=0
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272491    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b98778d74-pfhxh"
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b98778d74-pfhxh" event={"ID":"ce7c9a99-ca74-4c17-ab04-4d6d61d18384","Type":"ContainerDied","Data":"536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425"}
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272561    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b98778d74-pfhxh" event={"ID":"ce7c9a99-ca74-4c17-ab04-4d6d61d18384","Type":"ContainerDied","Data":"2ac56731cea37d73be6e6830f20bff2c8766b1c5e62bfc651349e3a29e4c6ebb"}
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272589    2564 scope.go:117] "RemoveContainer" containerID="536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425"
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272752    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272776    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd7hr\" (UniqueName: \"kubernetes.io/projected/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-kube-api-access-cd7hr\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272791    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272804    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272815    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:15.272900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.272829    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce7c9a99-ca74-4c17-ab04-4d6d61d18384-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:31:15.281554 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.281534    2564 scope.go:117] "RemoveContainer" containerID="536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425"
Jul 15 19:31:15.281805 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:31:15.281788    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425\": container with ID starting with 536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425 not found: ID does not exist" containerID="536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425"
Jul 15 19:31:15.281844 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.281813    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425"} err="failed to get container status \"536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425\": rpc error: code = NotFound desc = could not find container \"536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425\": container with ID starting with 536cc7674d327a1af0627416c60f73afc89305bbf5600f0d8263399d37008425 not found: ID does not exist"
Jul 15 19:31:15.290913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.290886    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b98778d74-pfhxh"]
Jul 15 19:31:15.294780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:15.294758    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b98778d74-pfhxh"]
Jul 15 19:31:16.491977 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:16.491944    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce7c9a99-ca74-4c17-ab04-4d6d61d18384" path="/var/lib/kubelet/pods/ce7c9a99-ca74-4c17-ab04-4d6d61d18384/volumes"
Jul 15 19:31:17.765446 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:17.765410    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:31:17.769637 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:17.769611    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8545cf9cdb-tvdjz"
Jul 15 19:31:19.314166 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:19.314126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:50.483782 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:50.483738    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:52.268581 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:52.268543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:53.303826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:53.303780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:55.589604 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589569    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589875    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b11b6f-c402-4307-919c-add8df8f4538" containerName="console"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589887    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b11b6f-c402-4307-919c-add8df8f4538" containerName="console"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589898    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" containerName="registry"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589903    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" containerName="registry"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589912    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce7c9a99-ca74-4c17-ab04-4d6d61d18384" containerName="console"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589918    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce7c9a99-ca74-4c17-ab04-4d6d61d18384" containerName="console"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589971    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="46844f3e-e443-47a1-a487-82d143fed4c9" containerName="registry"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589980    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce7c9a99-ca74-4c17-ab04-4d6d61d18384" containerName="console"
Jul 15 19:31:55.590010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.589987    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b11b6f-c402-4307-919c-add8df8f4538" containerName="console"
Jul 15 19:31:55.592782 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.592761    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.594877 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.594851    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t29m4\""
Jul 15 19:31:55.595065 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595047    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 19:31:55.595192 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595171    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 19:31:55.595353 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595313    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 19:31:55.595353 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 19:31:55.595523 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595374    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 19:31:55.595523 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595321    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 19:31:55.595523 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595273    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 19:31:55.595523 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.595429    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 19:31:55.602818 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.602798    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 19:31:55.607687 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.607661    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:31:55.684282 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684282 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-config-out\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chgrg\" (UniqueName: \"kubernetes.io/projected/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-kube-api-access-chgrg\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-config-volume\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684732 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-web-config\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684732 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684732 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684550    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.684732 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.684570    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785114 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chgrg\" (UniqueName: \"kubernetes.io/projected/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-kube-api-access-chgrg\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-config-volume\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785257    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-web-config\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785404    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-config-out\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.785983 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.786036 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.785981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.788453 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.788325    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-config-out\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.788652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.788691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.788741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-web-config\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.788837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789002 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.788949    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.789003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.789284    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.789454 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.789435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-config-volume\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.790681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.790662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.793019 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.793003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chgrg\" (UniqueName: \"kubernetes.io/projected/b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1-kube-api-access-chgrg\") pod \"alertmanager-main-0\" (UID: \"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:55.904252 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:55.904142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:31:56.041568 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:56.041546    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:31:56.043785 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:31:56.043754    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb028c2e6_aea2_47ef_a8f4_f1fe0d1b7fc1.slice/crio-2efc3c4778b951b7a585352a4d325ee81811955a073510affe92f70df821c1e7 WatchSource:0}: Error finding container 2efc3c4778b951b7a585352a4d325ee81811955a073510affe92f70df821c1e7: Status 404 returned error can't find the container with id 2efc3c4778b951b7a585352a4d325ee81811955a073510affe92f70df821c1e7
Jul 15 19:31:56.428436 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:56.428396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"2efc3c4778b951b7a585352a4d325ee81811955a073510affe92f70df821c1e7"}
Jul 15 19:31:57.339437 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.339344    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-686dff948d-fhvcd"]
Jul 15 19:31:57.341924 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.341896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.345118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.345079    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 15 19:31:57.345285 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.345147    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 15 19:31:57.345285 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.345152    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 15 19:31:57.345285 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.345238    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9bwqx\""
Jul 15 19:31:57.345440 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.345330    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 15 19:31:57.345489 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.345465    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 15 19:31:57.349716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.349682    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 15 19:31:57.353405 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.353298    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-686dff948d-fhvcd"]
Jul 15 19:31:57.397050 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-metrics-client-ca\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-federate-client-tls\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-secret-telemeter-client\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-serving-certs-ca-bundle\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397145    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxfh\" (UniqueName: \"kubernetes.io/projected/fe4d0f7b-727a-4746-83e8-4822fc359b09-kube-api-access-9dxfh\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.397424 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.397244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-telemeter-client-tls\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.433519 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.433479    2564 generic.go:356] "Generic (PLEG): container finished" podID="b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1" containerID="fd9d58bca3bb432eb99074b046181d0e8bc69a9ed80149775042b11bb4c755fe" exitCode=0
Jul 15 19:31:57.433519 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.433523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerDied","Data":"fd9d58bca3bb432eb99074b046181d0e8bc69a9ed80149775042b11bb4c755fe"}
Jul 15 19:31:57.498562 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.498759 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-metrics-client-ca\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.498759 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-federate-client-tls\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.498884 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-secret-telemeter-client\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.498938 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-serving-certs-ca-bundle\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.498938 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.499034 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.498964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxfh\" (UniqueName: \"kubernetes.io/projected/fe4d0f7b-727a-4746-83e8-4822fc359b09-kube-api-access-9dxfh\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.499091 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.499031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-telemeter-client-tls\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.499801 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.499772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-serving-certs-ca-bundle\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.499963 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.499937    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.500191 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.500165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe4d0f7b-727a-4746-83e8-4822fc359b09-metrics-client-ca\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.503060 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.501931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-federate-client-tls\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.503060 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.502271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-telemeter-client-tls\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.503060 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.502677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.503346 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.503325    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe4d0f7b-727a-4746-83e8-4822fc359b09-secret-telemeter-client\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.506384 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.506363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxfh\" (UniqueName: \"kubernetes.io/projected/fe4d0f7b-727a-4746-83e8-4822fc359b09-kube-api-access-9dxfh\") pod \"telemeter-client-686dff948d-fhvcd\" (UID: \"fe4d0f7b-727a-4746-83e8-4822fc359b09\") " pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.656116 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.656024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd"
Jul 15 19:31:57.808855 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:57.808830    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-686dff948d-fhvcd"]
Jul 15 19:31:57.810794 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:31:57.810767    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4d0f7b_727a_4746_83e8_4822fc359b09.slice/crio-21abaadad44c91e91d83f281a75871db726912425a19ccff40ff467122e2169f WatchSource:0}: Error finding container 21abaadad44c91e91d83f281a75871db726912425a19ccff40ff467122e2169f: Status 404 returned error can't find the container with id 21abaadad44c91e91d83f281a75871db726912425a19ccff40ff467122e2169f
Jul 15 19:31:58.441048 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:58.441002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd" event={"ID":"fe4d0f7b-727a-4746-83e8-4822fc359b09","Type":"ContainerStarted","Data":"21abaadad44c91e91d83f281a75871db726912425a19ccff40ff467122e2169f"}
Jul 15 19:31:59.447354 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:31:59.447260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"2614f51e3e8e1270d84bcc8c2b0701319dbad5ac8ae782fc18bc58b6b4971f8f"}
Jul 15 19:32:00.459544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.459497    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"b028c4d43ec2088318a164a0393bc5fd3dc195788e46f004a30a4ec1f96736e3"}
Jul 15 19:32:00.460038 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.459623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"bcd21983a029645da36ab5b7bfd5a58d5e00896776ce25a40cbc951696b9001f"}
Jul 15 19:32:00.460038 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.459665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"091172c7861136f12252da4f4bc6be248f74db4f6f3ded7a19df31120dadae9f"}
Jul 15 19:32:00.460038 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.459702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"31d633f14855d5ae049ff62299cac1f088962aa233da8698fdd12b4645d2c4a6"}
Jul 15 19:32:00.460038 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.459744    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1","Type":"ContainerStarted","Data":"a964f8c7f2dbdb70a8a94a6d717c42d014861f101b335a286265235c7252f763"}
Jul 15 19:32:00.461516 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.461490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd" event={"ID":"fe4d0f7b-727a-4746-83e8-4822fc359b09","Type":"ContainerStarted","Data":"6bfbcbfbaca73f5c0534ba4a524e136515a7f7b8dc42008f0175c0c745174675"}
Jul 15 19:32:00.461636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.461538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd" event={"ID":"fe4d0f7b-727a-4746-83e8-4822fc359b09","Type":"ContainerStarted","Data":"0a8ba28267243b701dd0d6fba7dcb7a4df1cb6099ddf5d37799e8539c04c1a49"}
Jul 15 19:32:00.461636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.461566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd" event={"ID":"fe4d0f7b-727a-4746-83e8-4822fc359b09","Type":"ContainerStarted","Data":"cdef8dd52ff5cc41fe90e3c4154fa47d83739e369648460de4666b38e49a8efd"}
Jul 15 19:32:00.482308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.482254    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.9149733920000003 podStartE2EDuration="5.48224065s" totalImagesPullingTime="2.552506123s" totalInitContainerRuntime="14.761135ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:31:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:00.481149785 +0000 UTC m=+210.558942587" watchObservedRunningTime="2026-07-15 19:32:00.48224065 +0000 UTC m=+210.560033442"
Jul 15 19:32:00.499853 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:00.499796    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-686dff948d-fhvcd" podStartSLOduration=1.74624216 podStartE2EDuration="3.499774383s" totalImagesPullingTime="1.753532223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:31:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:00.498000982 +0000 UTC m=+210.575793784" watchObservedRunningTime="2026-07-15 19:32:00.499774383 +0000 UTC m=+210.577567185"
Jul 15 19:32:01.265405 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.265371    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-887d6d8-77656"]
Jul 15 19:32:01.267997 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.267979    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.279461 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.279439    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-887d6d8-77656"]
Jul 15 19:32:01.318474 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.318443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:01.331518 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-service-ca\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.331780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-oauth-config\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.331780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-oauth-serving-cert\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.331780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331637    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-config\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.331780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331674    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-trusted-ca-bundle\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.331780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-serving-cert\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.331780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.331731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqhv5\" (UniqueName: \"kubernetes.io/projected/4a1fb17f-2f15-45d6-8e1b-0d137da93839-kube-api-access-qqhv5\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.419728 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.419679    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:01.432966 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.432923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-oauth-config\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.433172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.432980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-oauth-serving-cert\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.433172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.433033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-config\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.433172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.433060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-trusted-ca-bundle\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.433172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.433100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-serving-cert\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.433172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.433125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqhv5\" (UniqueName: \"kubernetes.io/projected/4a1fb17f-2f15-45d6-8e1b-0d137da93839-kube-api-access-qqhv5\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.433172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.433169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-service-ca\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.434152 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.434013    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-service-ca\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.434373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.434347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-trusted-ca-bundle\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.434613 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.434583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-oauth-serving-cert\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.434914 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.434896    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-config\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.436254 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.436236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-serving-cert\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.436704 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.436687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-oauth-config\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.441592 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.441571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqhv5\" (UniqueName: \"kubernetes.io/projected/4a1fb17f-2f15-45d6-8e1b-0d137da93839-kube-api-access-qqhv5\") pod \"console-887d6d8-77656\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") " pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.519622 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.519512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:01.579289 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.579240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:01.729908 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:01.729880    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-887d6d8-77656"]
Jul 15 19:32:01.734468 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:32:01.734438    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1fb17f_2f15_45d6_8e1b_0d137da93839.slice/crio-614468b8489db11fdf10324010a939cc9cdb57c81d261805a822cb310ffb68a8 WatchSource:0}: Error finding container 614468b8489db11fdf10324010a939cc9cdb57c81d261805a822cb310ffb68a8: Status 404 returned error can't find the container with id 614468b8489db11fdf10324010a939cc9cdb57c81d261805a822cb310ffb68a8
Jul 15 19:32:02.229150 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:02.229118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:02.472054 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:02.472011    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-887d6d8-77656" event={"ID":"4a1fb17f-2f15-45d6-8e1b-0d137da93839","Type":"ContainerStarted","Data":"da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439"}
Jul 15 19:32:02.472215 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:02.472078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-887d6d8-77656" event={"ID":"4a1fb17f-2f15-45d6-8e1b-0d137da93839","Type":"ContainerStarted","Data":"614468b8489db11fdf10324010a939cc9cdb57c81d261805a822cb310ffb68a8"}
Jul 15 19:32:02.487419 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:02.487318    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-887d6d8-77656" podStartSLOduration=1.487298924 podStartE2EDuration="1.487298924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:02.486583223 +0000 UTC m=+212.564376025" watchObservedRunningTime="2026-07-15 19:32:02.487298924 +0000 UTC m=+212.565091723"
Jul 15 19:32:11.579419 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:11.579373    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:11.579798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:11.579451    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:11.584092 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:11.584070    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:12.515787 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:12.515758    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:32:12.556817 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:12.556777    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797894dcc7-q9bnn"]
Jul 15 19:32:12.659408 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:12.659365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:12.761041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:12.761003    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:12.812882 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:12.812791    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:28.969588 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:28.969549    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:29.071671 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:29.071622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:37.582918 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.582855    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-797894dcc7-q9bnn" podUID="6a87c8c0-4f74-4d6d-a270-3c8037aef389" containerName="console" containerID="cri-o://c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e" gracePeriod=15
Jul 15 19:32:37.828708 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.828686    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:32:37.955370 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955341    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-oauth-serving-cert\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-serving-cert\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955411    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-trusted-ca-bundle\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955432    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-service-ca\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955718 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955569    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-config\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955718 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44tq4\" (UniqueName: \"kubernetes.io/projected/6a87c8c0-4f74-4d6d-a270-3c8037aef389-kube-api-access-44tq4\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955725    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-oauth-config\") pod \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\" (UID: \"6a87c8c0-4f74-4d6d-a270-3c8037aef389\") "
Jul 15 19:32:37.955865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955837    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:32:37.955865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955847    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:32:37.956019 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955873    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:32:37.956019 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.955916    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-config" (OuterVolumeSpecName: "console-config") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:32:37.956124 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.956039    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:37.956124 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.956060    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:37.956124 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.956074    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:37.956124 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.956087    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:37.957833 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.957813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:32:37.957998 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.957978    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a87c8c0-4f74-4d6d-a270-3c8037aef389-kube-api-access-44tq4" (OuterVolumeSpecName: "kube-api-access-44tq4") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "kube-api-access-44tq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:32:37.958062 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:37.958036    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a87c8c0-4f74-4d6d-a270-3c8037aef389" (UID: "6a87c8c0-4f74-4d6d-a270-3c8037aef389"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:32:38.056680 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.056637    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44tq4\" (UniqueName: \"kubernetes.io/projected/6a87c8c0-4f74-4d6d-a270-3c8037aef389-kube-api-access-44tq4\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:38.056680 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.056672    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:38.056680 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.056681    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a87c8c0-4f74-4d6d-a270-3c8037aef389-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:32:38.617985 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.617951    2564 generic.go:356] "Generic (PLEG): container finished" podID="6a87c8c0-4f74-4d6d-a270-3c8037aef389" containerID="c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e" exitCode=0
Jul 15 19:32:38.618406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.617997    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797894dcc7-q9bnn"
Jul 15 19:32:38.618406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.618046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797894dcc7-q9bnn" event={"ID":"6a87c8c0-4f74-4d6d-a270-3c8037aef389","Type":"ContainerDied","Data":"c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e"}
Jul 15 19:32:38.618406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.618098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797894dcc7-q9bnn" event={"ID":"6a87c8c0-4f74-4d6d-a270-3c8037aef389","Type":"ContainerDied","Data":"df3a7c7b8543d3588bf7fb38d654dbd1f144fccee86906c4ecf9d9517fbab201"}
Jul 15 19:32:38.618406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.618118    2564 scope.go:117] "RemoveContainer" containerID="c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e"
Jul 15 19:32:38.626684 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.626669    2564 scope.go:117] "RemoveContainer" containerID="c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e"
Jul 15 19:32:38.626952 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:32:38.626934    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e\": container with ID starting with c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e not found: ID does not exist" containerID="c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e"
Jul 15 19:32:38.627032 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.626960    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e"} err="failed to get container status \"c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e\": rpc error: code = NotFound desc = could not find container \"c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e\": container with ID starting with c8c560a6f5d7bc502ebfe4c8b910c8f503b4ebef82fe27ec1935c6ecf8be770e not found: ID does not exist"
Jul 15 19:32:38.632675 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.632652    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797894dcc7-q9bnn"]
Jul 15 19:32:38.636184 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:38.636163    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-797894dcc7-q9bnn"]
Jul 15 19:32:40.492309 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:32:40.492279    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a87c8c0-4f74-4d6d-a270-3c8037aef389" path="/var/lib/kubelet/pods/6a87c8c0-4f74-4d6d-a270-3c8037aef389/volumes"
Jul 15 19:33:07.507650 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:07.507612    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:07.565977 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:07.565940    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:16.089320 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.089290    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84795c6465-srd5s"]
Jul 15 19:33:16.089750 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.089639    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a87c8c0-4f74-4d6d-a270-3c8037aef389" containerName="console"
Jul 15 19:33:16.089750 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.089651    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a87c8c0-4f74-4d6d-a270-3c8037aef389" containerName="console"
Jul 15 19:33:16.089750 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.089697    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a87c8c0-4f74-4d6d-a270-3c8037aef389" containerName="console"
Jul 15 19:33:16.093829 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.093810    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.105591 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.105565    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84795c6465-srd5s"]
Jul 15 19:33:16.178144 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-oauth-config\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.178144 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178142    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-console-config\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.178413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178161    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-oauth-serving-cert\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.178413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-serving-cert\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.178413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-service-ca\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.178413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-trusted-ca-bundle\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.178413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.178352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bf2c\" (UniqueName: \"kubernetes.io/projected/3286494f-1767-4ce2-995b-9d586704d03e-kube-api-access-8bf2c\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.276343 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.276299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:16.279004 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.278984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-serving-cert\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279119 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-service-ca\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279119 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-trusted-ca-bundle\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279119 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bf2c\" (UniqueName: \"kubernetes.io/projected/3286494f-1767-4ce2-995b-9d586704d03e-kube-api-access-8bf2c\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279258 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279153    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-oauth-config\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279258 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-console-config\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279258 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-oauth-serving-cert\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.279999 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.279975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-service-ca\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.280099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.280024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-oauth-serving-cert\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.280328 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.280310    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-console-config\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.280373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.280342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-trusted-ca-bundle\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.281805 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.281782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-oauth-config\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.281950 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.281932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-serving-cert\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.286533 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.286510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bf2c\" (UniqueName: \"kubernetes.io/projected/3286494f-1767-4ce2-995b-9d586704d03e-kube-api-access-8bf2c\") pod \"console-84795c6465-srd5s\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") " pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.327808 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.327770    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:16.403641 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.403547    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:16.535079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.535057    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84795c6465-srd5s"]
Jul 15 19:33:16.537534 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:33:16.537499    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3286494f_1767_4ce2_995b_9d586704d03e.slice/crio-070bcff8243ea6de835924e0902927339c81b9a6f297e91a6e1a1b714571c5ad WatchSource:0}: Error finding container 070bcff8243ea6de835924e0902927339c81b9a6f297e91a6e1a1b714571c5ad: Status 404 returned error can't find the container with id 070bcff8243ea6de835924e0902927339c81b9a6f297e91a6e1a1b714571c5ad
Jul 15 19:33:16.769232 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.769189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84795c6465-srd5s" event={"ID":"3286494f-1767-4ce2-995b-9d586704d03e","Type":"ContainerStarted","Data":"e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff"}
Jul 15 19:33:16.769428 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.769260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84795c6465-srd5s" event={"ID":"3286494f-1767-4ce2-995b-9d586704d03e","Type":"ContainerStarted","Data":"070bcff8243ea6de835924e0902927339c81b9a6f297e91a6e1a1b714571c5ad"}
Jul 15 19:33:16.785435 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:16.785377    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84795c6465-srd5s" podStartSLOduration=0.785357103 podStartE2EDuration="785.357103ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:33:16.783987679 +0000 UTC m=+286.861780471" watchObservedRunningTime="2026-07-15 19:33:16.785357103 +0000 UTC m=+286.863149906"
Jul 15 19:33:19.113951 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:19.113890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:19.706135 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:19.706093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:19.808177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:19.808140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:23.341853 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:23.341818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:23.442746 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:23.442713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:23.495559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:23.495524    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:26.404373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:26.404333    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:26.404793 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:26.404413    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:26.408991 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:26.408967    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:26.814439 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:26.814409    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:33:26.857110 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:26.857071    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-887d6d8-77656"]
Jul 15 19:33:26.957894 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:26.957860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:27.007187 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:27.007150    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:27.057549 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:27.057504    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:30.350089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:30.350060    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:33:30.356557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:30.356534    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 19:33:30.357185 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:30.357166    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:33:51.880078 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:51.880033    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-887d6d8-77656" podUID="4a1fb17f-2f15-45d6-8e1b-0d137da93839" containerName="console" containerID="cri-o://da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439" gracePeriod=15
Jul 15 19:33:52.132291 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.132212    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:33:52.182557 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.182519    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-service-ca\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.182752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.182585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-serving-cert\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.182752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.182622    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-trusted-ca-bundle\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.182752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.182731    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-config\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.182911 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.182769    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-oauth-serving-cert\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.182911 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.182805    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-oauth-config\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.183734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.183428    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:33:52.183734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.183439    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:33:52.183734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.183478    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-config" (OuterVolumeSpecName: "console-config") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:33:52.183734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.183481    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqhv5\" (UniqueName: \"kubernetes.io/projected/4a1fb17f-2f15-45d6-8e1b-0d137da93839-kube-api-access-qqhv5\") pod \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\" (UID: \"4a1fb17f-2f15-45d6-8e1b-0d137da93839\") "
Jul 15 19:33:52.183734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.183679    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:33:52.185548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.184585    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.185548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.184620    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.185548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.184636    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.185548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.184650    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a1fb17f-2f15-45d6-8e1b-0d137da93839-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.187544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.187518    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a1fb17f-2f15-45d6-8e1b-0d137da93839-kube-api-access-qqhv5" (OuterVolumeSpecName: "kube-api-access-qqhv5") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "kube-api-access-qqhv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:33:52.188020 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.187992    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:33:52.190023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.189997    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a1fb17f-2f15-45d6-8e1b-0d137da93839" (UID: "4a1fb17f-2f15-45d6-8e1b-0d137da93839"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:33:52.285213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.285175    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.285213 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.285212    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqhv5\" (UniqueName: \"kubernetes.io/projected/4a1fb17f-2f15-45d6-8e1b-0d137da93839-kube-api-access-qqhv5\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.285441 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.285251    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a1fb17f-2f15-45d6-8e1b-0d137da93839-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:33:52.920857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.920767    2564 generic.go:356] "Generic (PLEG): container finished" podID="4a1fb17f-2f15-45d6-8e1b-0d137da93839" containerID="da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439" exitCode=0
Jul 15 19:33:52.920857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.920830    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-887d6d8-77656"
Jul 15 19:33:52.920857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.920834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-887d6d8-77656" event={"ID":"4a1fb17f-2f15-45d6-8e1b-0d137da93839","Type":"ContainerDied","Data":"da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439"}
Jul 15 19:33:52.921332 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.920883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-887d6d8-77656" event={"ID":"4a1fb17f-2f15-45d6-8e1b-0d137da93839","Type":"ContainerDied","Data":"614468b8489db11fdf10324010a939cc9cdb57c81d261805a822cb310ffb68a8"}
Jul 15 19:33:52.921332 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.920905    2564 scope.go:117] "RemoveContainer" containerID="da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439"
Jul 15 19:33:52.929328 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.929311    2564 scope.go:117] "RemoveContainer" containerID="da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439"
Jul 15 19:33:52.929595 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:33:52.929573    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439\": container with ID starting with da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439 not found: ID does not exist" containerID="da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439"
Jul 15 19:33:52.929649 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.929602    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439"} err="failed to get container status \"da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439\": rpc error: code = NotFound desc = could not find container \"da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439\": container with ID starting with da1ccf50ed7a8ec1595f224c3505e32fee765032a265a7674d103b6eb2dbf439 not found: ID does not exist"
Jul 15 19:33:52.936469 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.936447    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-887d6d8-77656"]
Jul 15 19:33:52.939743 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:52.939721    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-887d6d8-77656"]
Jul 15 19:33:54.492432 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:54.492392    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a1fb17f-2f15-45d6-8e1b-0d137da93839" path="/var/lib/kubelet/pods/4a1fb17f-2f15-45d6-8e1b-0d137da93839/volumes"
Jul 15 19:33:55.776811 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:55.776780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:55.877423 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:55.877383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:56.375809 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.375773    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"]
Jul 15 19:33:56.376134 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.376120    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a1fb17f-2f15-45d6-8e1b-0d137da93839" containerName="console"
Jul 15 19:33:56.376176 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.376137    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a1fb17f-2f15-45d6-8e1b-0d137da93839" containerName="console"
Jul 15 19:33:56.376209 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.376185    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a1fb17f-2f15-45d6-8e1b-0d137da93839" containerName="console"
Jul 15 19:33:56.380395 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.380379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.382390 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.382364    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 19:33:56.383103 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.383086    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 19:33:56.383165 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.383118    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-khlsp\""
Jul 15 19:33:56.386665 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.386639    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"]
Jul 15 19:33:56.418274 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.418246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.418405 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.418329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tff5k\" (UniqueName: \"kubernetes.io/projected/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-kube-api-access-tff5k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.418405 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.418354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.519522 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.519494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.519693 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.519555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tff5k\" (UniqueName: \"kubernetes.io/projected/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-kube-api-access-tff5k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.519693 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.519578    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.519892 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.519873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.519932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.519900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.527460 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.527429    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tff5k\" (UniqueName: \"kubernetes.io/projected/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-kube-api-access-tff5k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.691151 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.691058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:33:56.820964 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.820941    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"]
Jul 15 19:33:56.822817 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:33:56.822784    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd50dbc5_9cb7_40d2_9ede_28d3acd8194d.slice/crio-5c911875a4759dc5b0804a328a3f2e87af5176b84588f65e97098ac3e0ef4ca1 WatchSource:0}: Error finding container 5c911875a4759dc5b0804a328a3f2e87af5176b84588f65e97098ac3e0ef4ca1: Status 404 returned error can't find the container with id 5c911875a4759dc5b0804a328a3f2e87af5176b84588f65e97098ac3e0ef4ca1
Jul 15 19:33:56.824900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.824879    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:33:56.943669 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:33:56.943582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx" event={"ID":"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d","Type":"ContainerStarted","Data":"5c911875a4759dc5b0804a328a3f2e87af5176b84588f65e97098ac3e0ef4ca1"}
Jul 15 19:34:02.972324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:02.972287    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerID="f4244314803487c9f1fe7076abfc460e0ecfe8148f869b30ecf5a7e7397a72ba" exitCode=0
Jul 15 19:34:02.972796 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:02.972382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx" event={"ID":"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d","Type":"ContainerDied","Data":"f4244314803487c9f1fe7076abfc460e0ecfe8148f869b30ecf5a7e7397a72ba"}
Jul 15 19:34:05.987847 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:05.987812    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerID="2ad84b34ab563757c8922703065045b94e26e5c31f2c585dad96147ab98db443" exitCode=0
Jul 15 19:34:05.988201 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:05.987893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx" event={"ID":"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d","Type":"ContainerDied","Data":"2ad84b34ab563757c8922703065045b94e26e5c31f2c585dad96147ab98db443"}
Jul 15 19:34:13.023369 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:13.023332    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerID="92baedff67fe8a4eae664c50bf185280b4b2b3b90220804d570626a2dce6bffd" exitCode=0
Jul 15 19:34:13.023735 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:13.023425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx" event={"ID":"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d","Type":"ContainerDied","Data":"92baedff67fe8a4eae664c50bf185280b4b2b3b90220804d570626a2dce6bffd"}
Jul 15 19:34:14.203814 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.203781    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:34:14.278952 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.278856    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tff5k\" (UniqueName: \"kubernetes.io/projected/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-kube-api-access-tff5k\") pod \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") "
Jul 15 19:34:14.278952 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.278924    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-bundle\") pod \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") "
Jul 15 19:34:14.279155 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.279006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-util\") pod \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\" (UID: \"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d\") "
Jul 15 19:34:14.279652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.279622    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-bundle" (OuterVolumeSpecName: "bundle") pod "bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" (UID: "bd50dbc5-9cb7-40d2-9ede-28d3acd8194d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:14.281300 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.281273    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-kube-api-access-tff5k" (OuterVolumeSpecName: "kube-api-access-tff5k") pod "bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" (UID: "bd50dbc5-9cb7-40d2-9ede-28d3acd8194d"). InnerVolumeSpecName "kube-api-access-tff5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:34:14.283310 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.283284    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-util" (OuterVolumeSpecName: "util") pod "bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" (UID: "bd50dbc5-9cb7-40d2-9ede-28d3acd8194d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:14.379662 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.379622    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tff5k\" (UniqueName: \"kubernetes.io/projected/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-kube-api-access-tff5k\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:14.379662 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.379654    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:14.379662 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:14.379666    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd50dbc5-9cb7-40d2-9ede-28d3acd8194d-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:15.037882 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:15.037848    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx"
Jul 15 19:34:15.038077 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:15.037848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zqjzx" event={"ID":"bd50dbc5-9cb7-40d2-9ede-28d3acd8194d","Type":"ContainerDied","Data":"5c911875a4759dc5b0804a328a3f2e87af5176b84588f65e97098ac3e0ef4ca1"}
Jul 15 19:34:15.038077 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:15.037993    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c911875a4759dc5b0804a328a3f2e87af5176b84588f65e97098ac3e0ef4ca1"
Jul 15 19:34:17.334792 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:17.334746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:19.284818 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.284780    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"]
Jul 15 19:34:19.285418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285261    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="util"
Jul 15 19:34:19.285418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285279    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="util"
Jul 15 19:34:19.285418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285310    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="extract"
Jul 15 19:34:19.285418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285318    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="extract"
Jul 15 19:34:19.285418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285334    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="pull"
Jul 15 19:34:19.285418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285343    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="pull"
Jul 15 19:34:19.285719 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.285428    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd50dbc5-9cb7-40d2-9ede-28d3acd8194d" containerName="extract"
Jul 15 19:34:19.288158 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.288136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.290472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.290452    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:34:19.290595 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.290504    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mtpns\""
Jul 15 19:34:19.290595 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.290560    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 15 19:34:19.303629 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.303604    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"]
Jul 15 19:34:19.422054 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.422014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06707a6b-533a-42d4-a136-13e63f796234-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6jf6v\" (UID: \"06707a6b-533a-42d4-a136-13e63f796234\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.422249 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.422104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxm7\" (UniqueName: \"kubernetes.io/projected/06707a6b-533a-42d4-a136-13e63f796234-kube-api-access-cvxm7\") pod \"cert-manager-operator-controller-manager-55c444bf86-6jf6v\" (UID: \"06707a6b-533a-42d4-a136-13e63f796234\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.523264 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.523196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06707a6b-533a-42d4-a136-13e63f796234-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6jf6v\" (UID: \"06707a6b-533a-42d4-a136-13e63f796234\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.523435 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.523295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxm7\" (UniqueName: \"kubernetes.io/projected/06707a6b-533a-42d4-a136-13e63f796234-kube-api-access-cvxm7\") pod \"cert-manager-operator-controller-manager-55c444bf86-6jf6v\" (UID: \"06707a6b-533a-42d4-a136-13e63f796234\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.523643 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.523624    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06707a6b-533a-42d4-a136-13e63f796234-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6jf6v\" (UID: \"06707a6b-533a-42d4-a136-13e63f796234\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.534934 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.534865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxm7\" (UniqueName: \"kubernetes.io/projected/06707a6b-533a-42d4-a136-13e63f796234-kube-api-access-cvxm7\") pod \"cert-manager-operator-controller-manager-55c444bf86-6jf6v\" (UID: \"06707a6b-533a-42d4-a136-13e63f796234\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.597041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.596992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"
Jul 15 19:34:19.731193 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.731156    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v"]
Jul 15 19:34:19.734268 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:34:19.734215    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06707a6b_533a_42d4_a136_13e63f796234.slice/crio-c88f17a5af4484b7212e323c5add7091f5c3cdd98dc02eb8252e2870f491c721 WatchSource:0}: Error finding container c88f17a5af4484b7212e323c5add7091f5c3cdd98dc02eb8252e2870f491c721: Status 404 returned error can't find the container with id c88f17a5af4484b7212e323c5add7091f5c3cdd98dc02eb8252e2870f491c721
Jul 15 19:34:19.863013 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:19.862914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:20.058351 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:20.058318    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v" event={"ID":"06707a6b-533a-42d4-a136-13e63f796234","Type":"ContainerStarted","Data":"c88f17a5af4484b7212e323c5add7091f5c3cdd98dc02eb8252e2870f491c721"}
Jul 15 19:34:20.943691 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:20.943649    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:20.995938 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:20.995903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:22.069154 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:22.069114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v" event={"ID":"06707a6b-533a-42d4-a136-13e63f796234","Type":"ContainerStarted","Data":"525bfea867ca62bf6253f24051f7c32109d97847a92b473f3159e4c14d69e698"}
Jul 15 19:34:22.092079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:22.092025    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6jf6v" podStartSLOduration=1.247607746 podStartE2EDuration="3.09200876s" totalImagesPullingTime="1.844401014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:34:22.090136209 +0000 UTC m=+352.167929010" watchObservedRunningTime="2026-07-15 19:34:22.09200876 +0000 UTC m=+352.169801561"
Jul 15 19:34:22.516373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:22.516322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:22.612130 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:22.612097    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:23.182897 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.182860    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"]
Jul 15 19:34:23.185759 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.185738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.188484 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.188466    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-khlsp\""
Jul 15 19:34:23.189176 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.189161    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 19:34:23.189295 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.189276    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 19:34:23.203897 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.203820    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"]
Jul 15 19:34:23.295565 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.295532    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:23.360734 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.360703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttnpt\" (UniqueName: \"kubernetes.io/projected/0cc5d052-784e-44c3-9201-910cbb889d11-kube-api-access-ttnpt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.360948 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.360758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.360948 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.360843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.397676 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.397639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:23.415791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.415761    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6s5g7"]
Jul 15 19:34:23.418346 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.418325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.420631 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.420610    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 19:34:23.420763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.420674    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-dnzts\""
Jul 15 19:34:23.420763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.420676    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 19:34:23.429743 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.429722    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6s5g7"]
Jul 15 19:34:23.462292 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.462264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttnpt\" (UniqueName: \"kubernetes.io/projected/0cc5d052-784e-44c3-9201-910cbb889d11-kube-api-access-ttnpt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.462428 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.462316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.462428 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.462344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.462655 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.462638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.462725 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.462708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.472012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.471979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttnpt\" (UniqueName: \"kubernetes.io/projected/0cc5d052-784e-44c3-9201-910cbb889d11-kube-api-access-ttnpt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.494894 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.494870    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:23.563385 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.563354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n5vb\" (UniqueName: \"kubernetes.io/projected/d52f8acc-ebde-48ca-a04d-62415cc3838a-kube-api-access-8n5vb\") pod \"cert-manager-webhook-6486ffd997-6s5g7\" (UID: \"d52f8acc-ebde-48ca-a04d-62415cc3838a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.563526 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.563394    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d52f8acc-ebde-48ca-a04d-62415cc3838a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6s5g7\" (UID: \"d52f8acc-ebde-48ca-a04d-62415cc3838a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.623933 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.623902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"]
Jul 15 19:34:23.625340 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:34:23.625305    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc5d052_784e_44c3_9201_910cbb889d11.slice/crio-0aadb142128b26135d4519a4b290614592110d950353d02e38cfa3a5e516a906 WatchSource:0}: Error finding container 0aadb142128b26135d4519a4b290614592110d950353d02e38cfa3a5e516a906: Status 404 returned error can't find the container with id 0aadb142128b26135d4519a4b290614592110d950353d02e38cfa3a5e516a906
Jul 15 19:34:23.664641 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.664608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n5vb\" (UniqueName: \"kubernetes.io/projected/d52f8acc-ebde-48ca-a04d-62415cc3838a-kube-api-access-8n5vb\") pod \"cert-manager-webhook-6486ffd997-6s5g7\" (UID: \"d52f8acc-ebde-48ca-a04d-62415cc3838a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.664777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.664649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d52f8acc-ebde-48ca-a04d-62415cc3838a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6s5g7\" (UID: \"d52f8acc-ebde-48ca-a04d-62415cc3838a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.671945 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.671918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d52f8acc-ebde-48ca-a04d-62415cc3838a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6s5g7\" (UID: \"d52f8acc-ebde-48ca-a04d-62415cc3838a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.672267 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.672214    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n5vb\" (UniqueName: \"kubernetes.io/projected/d52f8acc-ebde-48ca-a04d-62415cc3838a-kube-api-access-8n5vb\") pod \"cert-manager-webhook-6486ffd997-6s5g7\" (UID: \"d52f8acc-ebde-48ca-a04d-62415cc3838a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.742854 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.742829    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:23.872347 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:23.872323    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6s5g7"]
Jul 15 19:34:23.874251 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:34:23.874205    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd52f8acc_ebde_48ca_a04d_62415cc3838a.slice/crio-1878a58bf8c4919a7e3e342d0dd0c2ca3010bfa710a0c4367cf068bdb1485e8d WatchSource:0}: Error finding container 1878a58bf8c4919a7e3e342d0dd0c2ca3010bfa710a0c4367cf068bdb1485e8d: Status 404 returned error can't find the container with id 1878a58bf8c4919a7e3e342d0dd0c2ca3010bfa710a0c4367cf068bdb1485e8d
Jul 15 19:34:24.079396 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:24.079296    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cc5d052-784e-44c3-9201-910cbb889d11" containerID="fe49bf7ac04030f3e4ad46405cc67a0fcb8b4d5df61e248c0f6a1fcd46d81e8e" exitCode=0
Jul 15 19:34:24.079396 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:24.079383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr" event={"ID":"0cc5d052-784e-44c3-9201-910cbb889d11","Type":"ContainerDied","Data":"fe49bf7ac04030f3e4ad46405cc67a0fcb8b4d5df61e248c0f6a1fcd46d81e8e"}
Jul 15 19:34:24.079620 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:24.079449    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr" event={"ID":"0cc5d052-784e-44c3-9201-910cbb889d11","Type":"ContainerStarted","Data":"0aadb142128b26135d4519a4b290614592110d950353d02e38cfa3a5e516a906"}
Jul 15 19:34:24.080533 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:24.080511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7" event={"ID":"d52f8acc-ebde-48ca-a04d-62415cc3838a","Type":"ContainerStarted","Data":"1878a58bf8c4919a7e3e342d0dd0c2ca3010bfa710a0c4367cf068bdb1485e8d"}
Jul 15 19:34:25.502691 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:25.502644    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:25.691407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:25.691333    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:27.934156 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:27.934116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:28.035486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:28.035442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:28.104141 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:28.104103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7" event={"ID":"d52f8acc-ebde-48ca-a04d-62415cc3838a","Type":"ContainerStarted","Data":"6c81d4fd00fe39418f05de8201272238ad52b947dbb6057d865b74ce4450c6b3"}
Jul 15 19:34:28.104349 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:28.104156    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:28.105916 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:28.105890    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cc5d052-784e-44c3-9201-910cbb889d11" containerID="f90c290cd85b750c2d316047d3ae7e2f92b9ce170706aa69bca861fe06e0f54d" exitCode=0
Jul 15 19:34:28.106026 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:28.105930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr" event={"ID":"0cc5d052-784e-44c3-9201-910cbb889d11","Type":"ContainerDied","Data":"f90c290cd85b750c2d316047d3ae7e2f92b9ce170706aa69bca861fe06e0f54d"}
Jul 15 19:34:28.118488 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:28.118216    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7" podStartSLOduration=1.846210965 podStartE2EDuration="5.118199353s" totalImagesPullingTime="3.271988388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:34:28.117846161 +0000 UTC m=+358.195638961" watchObservedRunningTime="2026-07-15 19:34:28.118199353 +0000 UTC m=+358.195992156"
Jul 15 19:34:29.113085 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:29.113048    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cc5d052-784e-44c3-9201-910cbb889d11" containerID="1de65421a49b2a1fae9a4af5aa1ec474d9747dadb124dcda185aeb96f364a332" exitCode=0
Jul 15 19:34:29.113492 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:29.113131    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr" event={"ID":"0cc5d052-784e-44c3-9201-910cbb889d11","Type":"ContainerDied","Data":"1de65421a49b2a1fae9a4af5aa1ec474d9747dadb124dcda185aeb96f364a332"}
Jul 15 19:34:30.247014 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.246984    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:30.425589 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.425561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttnpt\" (UniqueName: \"kubernetes.io/projected/0cc5d052-784e-44c3-9201-910cbb889d11-kube-api-access-ttnpt\") pod \"0cc5d052-784e-44c3-9201-910cbb889d11\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") "
Jul 15 19:34:30.425733 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.425616    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-util\") pod \"0cc5d052-784e-44c3-9201-910cbb889d11\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") "
Jul 15 19:34:30.425804 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.425787    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-bundle\") pod \"0cc5d052-784e-44c3-9201-910cbb889d11\" (UID: \"0cc5d052-784e-44c3-9201-910cbb889d11\") "
Jul 15 19:34:30.426174 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.426137    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-bundle" (OuterVolumeSpecName: "bundle") pod "0cc5d052-784e-44c3-9201-910cbb889d11" (UID: "0cc5d052-784e-44c3-9201-910cbb889d11"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:30.427877 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.427843    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cc5d052-784e-44c3-9201-910cbb889d11-kube-api-access-ttnpt" (OuterVolumeSpecName: "kube-api-access-ttnpt") pod "0cc5d052-784e-44c3-9201-910cbb889d11" (UID: "0cc5d052-784e-44c3-9201-910cbb889d11"). InnerVolumeSpecName "kube-api-access-ttnpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:34:30.430408 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.430371    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-util" (OuterVolumeSpecName: "util") pod "0cc5d052-784e-44c3-9201-910cbb889d11" (UID: "0cc5d052-784e-44c3-9201-910cbb889d11"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:30.527183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.527153    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:30.527183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.527178    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttnpt\" (UniqueName: \"kubernetes.io/projected/0cc5d052-784e-44c3-9201-910cbb889d11-kube-api-access-ttnpt\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:30.527183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.527189    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cc5d052-784e-44c3-9201-910cbb889d11-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:30.744716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.744682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:30.843837 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:30.843800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:31.126363 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:31.126281    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr" event={"ID":"0cc5d052-784e-44c3-9201-910cbb889d11","Type":"ContainerDied","Data":"0aadb142128b26135d4519a4b290614592110d950353d02e38cfa3a5e516a906"}
Jul 15 19:34:31.126363 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:31.126330    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0aadb142128b26135d4519a4b290614592110d950353d02e38cfa3a5e516a906"
Jul 15 19:34:31.126363 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:31.126296    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhk5sr"
Jul 15 19:34:33.778108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:33.778071    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:34.116040 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:34.115943    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6s5g7"
Jul 15 19:34:35.348956 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.348915    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qvtqr"]
Jul 15 19:34:35.349477 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349457    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="util"
Jul 15 19:34:35.349551 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349481    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="util"
Jul 15 19:34:35.349551 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349492    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="pull"
Jul 15 19:34:35.349551 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349501    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="pull"
Jul 15 19:34:35.349551 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349514    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="extract"
Jul 15 19:34:35.349551 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349522    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="extract"
Jul 15 19:34:35.349789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.349614    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cc5d052-784e-44c3-9201-910cbb889d11" containerName="extract"
Jul 15 19:34:35.352586 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.352564    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.356622 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.356602    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f64dl\""
Jul 15 19:34:35.359436 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.359413    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chjfb\" (UniqueName: \"kubernetes.io/projected/1933c1af-67d2-4b13-ae89-fe3c31966a39-kube-api-access-chjfb\") pod \"cert-manager-55b4d954d5-qvtqr\" (UID: \"1933c1af-67d2-4b13-ae89-fe3c31966a39\") " pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.359536 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.359461    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1933c1af-67d2-4b13-ae89-fe3c31966a39-bound-sa-token\") pod \"cert-manager-55b4d954d5-qvtqr\" (UID: \"1933c1af-67d2-4b13-ae89-fe3c31966a39\") " pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.363500 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.363478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qvtqr"]
Jul 15 19:34:35.460167 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.460119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chjfb\" (UniqueName: \"kubernetes.io/projected/1933c1af-67d2-4b13-ae89-fe3c31966a39-kube-api-access-chjfb\") pod \"cert-manager-55b4d954d5-qvtqr\" (UID: \"1933c1af-67d2-4b13-ae89-fe3c31966a39\") " pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.460369 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.460184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1933c1af-67d2-4b13-ae89-fe3c31966a39-bound-sa-token\") pod \"cert-manager-55b4d954d5-qvtqr\" (UID: \"1933c1af-67d2-4b13-ae89-fe3c31966a39\") " pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.467823 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.467783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1933c1af-67d2-4b13-ae89-fe3c31966a39-bound-sa-token\") pod \"cert-manager-55b4d954d5-qvtqr\" (UID: \"1933c1af-67d2-4b13-ae89-fe3c31966a39\") " pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.467949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.467824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chjfb\" (UniqueName: \"kubernetes.io/projected/1933c1af-67d2-4b13-ae89-fe3c31966a39-kube-api-access-chjfb\") pod \"cert-manager-55b4d954d5-qvtqr\" (UID: \"1933c1af-67d2-4b13-ae89-fe3c31966a39\") " pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.661054 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.660966    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qvtqr"
Jul 15 19:34:35.791326 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:35.791289    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qvtqr"]
Jul 15 19:34:35.795323 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:34:35.795294    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1933c1af_67d2_4b13_ae89_fe3c31966a39.slice/crio-b1911925f251dbc5f493de104be0c7f7b9de3090b35bcf65ac08fb7b21df4ef4 WatchSource:0}: Error finding container b1911925f251dbc5f493de104be0c7f7b9de3090b35bcf65ac08fb7b21df4ef4: Status 404 returned error can't find the container with id b1911925f251dbc5f493de104be0c7f7b9de3090b35bcf65ac08fb7b21df4ef4
Jul 15 19:34:36.156882 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:36.156846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qvtqr" event={"ID":"1933c1af-67d2-4b13-ae89-fe3c31966a39","Type":"ContainerStarted","Data":"d7a4a2d103a549838329f6bee47c5ab2e7961781adac5a2e50bde65a455ea4ed"}
Jul 15 19:34:36.157084 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:36.156893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qvtqr" event={"ID":"1933c1af-67d2-4b13-ae89-fe3c31966a39","Type":"ContainerStarted","Data":"b1911925f251dbc5f493de104be0c7f7b9de3090b35bcf65ac08fb7b21df4ef4"}
Jul 15 19:34:36.174762 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:36.174714    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qvtqr" podStartSLOduration=1.174693466 podStartE2EDuration="1.174693466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:34:36.173506117 +0000 UTC m=+366.251298915" watchObservedRunningTime="2026-07-15 19:34:36.174693466 +0000 UTC m=+366.252486270"
Jul 15 19:34:39.350429 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:39.350392    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:39.450136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:39.450093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:46.933559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:46.933516    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:47.026327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:47.026284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:49.123521 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.123481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:49.807097 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.807060    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"]
Jul 15 19:34:49.812118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.812100    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.814163 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.814123    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-khlsp\""
Jul 15 19:34:49.814305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.814268    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 19:34:49.815039 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.814950    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 19:34:49.817494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.817474    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"]
Jul 15 19:34:49.879486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.879453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.879486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.879488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.879690 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.879508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htqgc\" (UniqueName: \"kubernetes.io/projected/835482dc-9fe1-4a8c-a4d6-90f56eafe244-kube-api-access-htqgc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.980502 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.980459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.980502 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.980503    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.980832 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.980523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqgc\" (UniqueName: \"kubernetes.io/projected/835482dc-9fe1-4a8c-a4d6-90f56eafe244-kube-api-access-htqgc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.980972 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.980951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.981033 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.980968    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:49.988373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:49.988349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqgc\" (UniqueName: \"kubernetes.io/projected/835482dc-9fe1-4a8c-a4d6-90f56eafe244-kube-api-access-htqgc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:50.123622 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:50.123529    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:50.253602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:50.253576    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"]
Jul 15 19:34:50.255592 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:34:50.255560    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835482dc_9fe1_4a8c_a4d6_90f56eafe244.slice/crio-ea45b6668dd3356b0a03769f5786ca97ed3b3b41a62b4f026ef06ea69955fad3 WatchSource:0}: Error finding container ea45b6668dd3356b0a03769f5786ca97ed3b3b41a62b4f026ef06ea69955fad3: Status 404 returned error can't find the container with id ea45b6668dd3356b0a03769f5786ca97ed3b3b41a62b4f026ef06ea69955fad3
Jul 15 19:34:51.225144 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:51.225110    2564 generic.go:356] "Generic (PLEG): container finished" podID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerID="cff7a34994cfc981703d42da289f239d7e33bda2312d35881c131dbd20eaeb79" exitCode=0
Jul 15 19:34:51.225580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:51.225208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444" event={"ID":"835482dc-9fe1-4a8c-a4d6-90f56eafe244","Type":"ContainerDied","Data":"cff7a34994cfc981703d42da289f239d7e33bda2312d35881c131dbd20eaeb79"}
Jul 15 19:34:51.225580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:51.225285    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444" event={"ID":"835482dc-9fe1-4a8c-a4d6-90f56eafe244","Type":"ContainerStarted","Data":"ea45b6668dd3356b0a03769f5786ca97ed3b3b41a62b4f026ef06ea69955fad3"}
Jul 15 19:34:52.232308 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:52.232279    2564 generic.go:356] "Generic (PLEG): container finished" podID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerID="4af0bf8cc837d6b37f7829a340019b13cc796e3e83d3440cb0e54f15d5ebd6fc" exitCode=0
Jul 15 19:34:52.232694 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:52.232376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444" event={"ID":"835482dc-9fe1-4a8c-a4d6-90f56eafe244","Type":"ContainerDied","Data":"4af0bf8cc837d6b37f7829a340019b13cc796e3e83d3440cb0e54f15d5ebd6fc"}
Jul 15 19:34:53.241352 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:53.241318    2564 generic.go:356] "Generic (PLEG): container finished" podID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerID="8da253d4a4fd80843b518f76c9ec4c29320a80b813bcb3a45115b0cef315df7e" exitCode=0
Jul 15 19:34:53.241712 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:53.241449    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444" event={"ID":"835482dc-9fe1-4a8c-a4d6-90f56eafe244","Type":"ContainerDied","Data":"8da253d4a4fd80843b518f76c9ec4c29320a80b813bcb3a45115b0cef315df7e"}
Jul 15 19:34:54.368944 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.368919    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:54.420920 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.420877    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-util\") pod \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") "
Jul 15 19:34:54.420920 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.420928    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-bundle\") pod \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") "
Jul 15 19:34:54.421160 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.421012    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htqgc\" (UniqueName: \"kubernetes.io/projected/835482dc-9fe1-4a8c-a4d6-90f56eafe244-kube-api-access-htqgc\") pod \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\" (UID: \"835482dc-9fe1-4a8c-a4d6-90f56eafe244\") "
Jul 15 19:34:54.421704 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.421678    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-bundle" (OuterVolumeSpecName: "bundle") pod "835482dc-9fe1-4a8c-a4d6-90f56eafe244" (UID: "835482dc-9fe1-4a8c-a4d6-90f56eafe244"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:54.423139 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.423114    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835482dc-9fe1-4a8c-a4d6-90f56eafe244-kube-api-access-htqgc" (OuterVolumeSpecName: "kube-api-access-htqgc") pod "835482dc-9fe1-4a8c-a4d6-90f56eafe244" (UID: "835482dc-9fe1-4a8c-a4d6-90f56eafe244"). InnerVolumeSpecName "kube-api-access-htqgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:34:54.426524 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.426497    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-util" (OuterVolumeSpecName: "util") pod "835482dc-9fe1-4a8c-a4d6-90f56eafe244" (UID: "835482dc-9fe1-4a8c-a4d6-90f56eafe244"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:54.521580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.521548    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:54.521580 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.521577    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/835482dc-9fe1-4a8c-a4d6-90f56eafe244-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:54.521749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:54.521589    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htqgc\" (UniqueName: \"kubernetes.io/projected/835482dc-9fe1-4a8c-a4d6-90f56eafe244-kube-api-access-htqgc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:34:55.252392 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:55.252363    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444"
Jul 15 19:34:55.252556 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:55.252357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnv444" event={"ID":"835482dc-9fe1-4a8c-a4d6-90f56eafe244","Type":"ContainerDied","Data":"ea45b6668dd3356b0a03769f5786ca97ed3b3b41a62b4f026ef06ea69955fad3"}
Jul 15 19:34:55.252556 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:34:55.252490    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea45b6668dd3356b0a03769f5786ca97ed3b3b41a62b4f026ef06ea69955fad3"
Jul 15 19:35:00.751137 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:00.751100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:00.854499 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:00.854465    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:01.378855 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.378819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:01.561336 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.561294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:01.666502 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.666411    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:01.682780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.682742    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"]
Jul 15 19:35:01.683269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683252    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="extract"
Jul 15 19:35:01.683324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683273    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="extract"
Jul 15 19:35:01.683324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683297    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="pull"
Jul 15 19:35:01.683324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683305    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="pull"
Jul 15 19:35:01.683416 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683327    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="util"
Jul 15 19:35:01.683416 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683337    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="util"
Jul 15 19:35:01.683478 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.683426    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="835482dc-9fe1-4a8c-a4d6-90f56eafe244" containerName="extract"
Jul 15 19:35:01.685674 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.685658    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.688675 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.688653    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 19:35:01.688804 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.688654    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w5j8n\""
Jul 15 19:35:01.688804 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.688654    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 19:35:01.689115 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.689101    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 19:35:01.692849 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.692829    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 19:35:01.704464 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.704436    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"]
Jul 15 19:35:01.783703 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.783671    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/989a7f1f-f58c-4170-863c-5e2a2802667a-webhook-cert\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.784082 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.783719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/989a7f1f-f58c-4170-863c-5e2a2802667a-manifests\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.784082 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.783780    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/989a7f1f-f58c-4170-863c-5e2a2802667a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.784082 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.783820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szx7c\" (UniqueName: \"kubernetes.io/projected/989a7f1f-f58c-4170-863c-5e2a2802667a-kube-api-access-szx7c\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.784082 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.783883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989a7f1f-f58c-4170-863c-5e2a2802667a-tmp\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.885290 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.885252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/989a7f1f-f58c-4170-863c-5e2a2802667a-webhook-cert\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.885501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.885307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/989a7f1f-f58c-4170-863c-5e2a2802667a-manifests\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.885501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.885326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/989a7f1f-f58c-4170-863c-5e2a2802667a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.885501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.885349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szx7c\" (UniqueName: \"kubernetes.io/projected/989a7f1f-f58c-4170-863c-5e2a2802667a-kube-api-access-szx7c\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.885501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.885422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989a7f1f-f58c-4170-863c-5e2a2802667a-tmp\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.885804 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.885769    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/989a7f1f-f58c-4170-863c-5e2a2802667a-manifests\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.888029 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.888000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989a7f1f-f58c-4170-863c-5e2a2802667a-tmp\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.888155 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.888135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/989a7f1f-f58c-4170-863c-5e2a2802667a-webhook-cert\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.888284 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.888266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/989a7f1f-f58c-4170-863c-5e2a2802667a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.893815 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.893790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szx7c\" (UniqueName: \"kubernetes.io/projected/989a7f1f-f58c-4170-863c-5e2a2802667a-kube-api-access-szx7c\") pod \"opendatahub-operator-controller-manager-58dd997468-wwljd\" (UID: \"989a7f1f-f58c-4170-863c-5e2a2802667a\") " pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:01.910329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.910291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:01.996170 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:01.996126    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:02.011914 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.011875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:02.111953 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.111900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:02.144426 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.144378    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"]
Jul 15 19:35:02.147518 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:35:02.147489    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989a7f1f_f58c_4170_863c_5e2a2802667a.slice/crio-206783e2218308a7ce3feb0ee907576176772d535f9654d32d3b48bd549a44a9 WatchSource:0}: Error finding container 206783e2218308a7ce3feb0ee907576176772d535f9654d32d3b48bd549a44a9: Status 404 returned error can't find the container with id 206783e2218308a7ce3feb0ee907576176772d535f9654d32d3b48bd549a44a9
Jul 15 19:35:02.212461 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.212427    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:02.284985 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.284881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd" event={"ID":"989a7f1f-f58c-4170-863c-5e2a2802667a","Type":"ContainerStarted","Data":"206783e2218308a7ce3feb0ee907576176772d535f9654d32d3b48bd549a44a9"}
Jul 15 19:35:02.315267 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.315216    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:02.416717 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:02.416678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:03.449018 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:03.448931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:03.540560 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:03.540520    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:06.309199 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:06.309155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd" event={"ID":"989a7f1f-f58c-4170-863c-5e2a2802667a","Type":"ContainerStarted","Data":"84ef74d9fc78d3a3022f673d99115213c104100ce420c3153b393b44c4aa5d7c"}
Jul 15 19:35:07.315406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:07.315369    2564 generic.go:356] "Generic (PLEG): container finished" podID="989a7f1f-f58c-4170-863c-5e2a2802667a" containerID="84ef74d9fc78d3a3022f673d99115213c104100ce420c3153b393b44c4aa5d7c" exitCode=0
Jul 15 19:35:07.315783 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:07.315459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd" event={"ID":"989a7f1f-f58c-4170-863c-5e2a2802667a","Type":"ContainerDied","Data":"84ef74d9fc78d3a3022f673d99115213c104100ce420c3153b393b44c4aa5d7c"}
Jul 15 19:35:08.322455 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:08.322419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd" event={"ID":"989a7f1f-f58c-4170-863c-5e2a2802667a","Type":"ContainerStarted","Data":"51ada54cda2da1d56a88f66eaed3213f9ebd173e6e250242ad4500890427f574"}
Jul 15 19:35:08.322848 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:08.322553    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:08.343068 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:08.343009    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd" podStartSLOduration=2.559060364 podStartE2EDuration="7.342991927s" totalImagesPullingTime="3.736644698s" totalInitContainerRuntime="1.047286865s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:08.340926692 +0000 UTC m=+398.418719490" watchObservedRunningTime="2026-07-15 19:35:08.342991927 +0000 UTC m=+398.420784727"
Jul 15 19:35:19.329705 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:19.329669    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58dd997468-wwljd"
Jul 15 19:35:20.877549 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:20.877499    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:21.013664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.013620    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:21.887869 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.887837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:21.972079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.972047    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"]
Jul 15 19:35:21.982481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.982458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:21.985706 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.985680    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 19:35:21.985853 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.985689    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 19:35:21.985853 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.985766    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 19:35:21.986138 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.986116    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:35:21.986253 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.986140    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xwxjr\""
Jul 15 19:35:21.986306 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.986119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 19:35:21.987844 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.987823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"]
Jul 15 19:35:21.988861 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:21.988837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:22.062214 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.062182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:22.065239 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.065205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1d799d6-1c80-4b78-b31f-21c2546b92e2-cert\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.065310 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.065270    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7bb5\" (UniqueName: \"kubernetes.io/projected/c1d799d6-1c80-4b78-b31f-21c2546b92e2-kube-api-access-h7bb5\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.065347 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.065323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1d799d6-1c80-4b78-b31f-21c2546b92e2-metrics-cert\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.065384 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.065364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c1d799d6-1c80-4b78-b31f-21c2546b92e2-manager-config\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.165932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.165848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7bb5\" (UniqueName: \"kubernetes.io/projected/c1d799d6-1c80-4b78-b31f-21c2546b92e2-kube-api-access-h7bb5\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.165932 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.165902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1d799d6-1c80-4b78-b31f-21c2546b92e2-metrics-cert\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.166119 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.165949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c1d799d6-1c80-4b78-b31f-21c2546b92e2-manager-config\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.166119 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.165970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1d799d6-1c80-4b78-b31f-21c2546b92e2-cert\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.166629 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.166601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c1d799d6-1c80-4b78-b31f-21c2546b92e2-manager-config\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.168660 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.168628    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1d799d6-1c80-4b78-b31f-21c2546b92e2-metrics-cert\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.168759 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.168668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1d799d6-1c80-4b78-b31f-21c2546b92e2-cert\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.174108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.174083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7bb5\" (UniqueName: \"kubernetes.io/projected/c1d799d6-1c80-4b78-b31f-21c2546b92e2-kube-api-access-h7bb5\") pod \"lws-controller-manager-7784c86d4c-ffnll\" (UID: \"c1d799d6-1c80-4b78-b31f-21c2546b92e2\") " pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.274086 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.274043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:22.293810 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.293781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:22.403066 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.403015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:22.429471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.429446    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"]
Jul 15 19:35:22.431621 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:35:22.431582    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d799d6_1c80_4b78_b31f_21c2546b92e2.slice/crio-0582ba99ccefcba4881ad6cebaf2f77921eb6e5839aac9a4fcfdb57eea2e5bc1 WatchSource:0}: Error finding container 0582ba99ccefcba4881ad6cebaf2f77921eb6e5839aac9a4fcfdb57eea2e5bc1: Status 404 returned error can't find the container with id 0582ba99ccefcba4881ad6cebaf2f77921eb6e5839aac9a4fcfdb57eea2e5bc1
Jul 15 19:35:22.568185 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:22.568146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:23.400726 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:23.400687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll" event={"ID":"c1d799d6-1c80-4b78-b31f-21c2546b92e2","Type":"ContainerStarted","Data":"0582ba99ccefcba4881ad6cebaf2f77921eb6e5839aac9a4fcfdb57eea2e5bc1"}
Jul 15 19:35:24.407971 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:24.407888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll" event={"ID":"c1d799d6-1c80-4b78-b31f-21c2546b92e2","Type":"ContainerStarted","Data":"8663ce08a8b0200eebfda8a97a6eca6748754ce2889337b9a5d84ffa2e75a152"}
Jul 15 19:35:24.408371 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:24.408024    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:24.424744 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:24.424696    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll" podStartSLOduration=1.796407262 podStartE2EDuration="3.424681241s" totalImagesPullingTime="1.628273979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:24.422657505 +0000 UTC m=+414.500450307" watchObservedRunningTime="2026-07-15 19:35:24.424681241 +0000 UTC m=+414.502474041"
Jul 15 19:35:26.876662 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.876618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:26.892101 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.892071    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"]
Jul 15 19:35:26.895524 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.895508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:26.897865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.897845    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 19:35:26.897928 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.897859    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hz6hl\""
Jul 15 19:35:26.897978 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.897848    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 19:35:26.907063 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:26.907038    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"]
Jul 15 19:35:27.010486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.010451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwrlv\" (UniqueName: \"kubernetes.io/projected/18631460-5a0f-4b5c-b7c8-92e09620ca63-kube-api-access-fwrlv\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.010657 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.010534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18631460-5a0f-4b5c-b7c8-92e09620ca63-tmp\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.010657 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.010560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18631460-5a0f-4b5c-b7c8-92e09620ca63-tls-certs\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.111445 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.111406    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18631460-5a0f-4b5c-b7c8-92e09620ca63-tmp\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.111609 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.111455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18631460-5a0f-4b5c-b7c8-92e09620ca63-tls-certs\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.111609 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.111517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwrlv\" (UniqueName: \"kubernetes.io/projected/18631460-5a0f-4b5c-b7c8-92e09620ca63-kube-api-access-fwrlv\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.111696 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:35:27.111618    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 15 19:35:27.111696 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:35:27.111678    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18631460-5a0f-4b5c-b7c8-92e09620ca63-tls-certs podName:18631460-5a0f-4b5c-b7c8-92e09620ca63 nodeName:}" failed. No retries permitted until 2026-07-15 19:35:27.611662062 +0000 UTC m=+417.689454844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/18631460-5a0f-4b5c-b7c8-92e09620ca63-tls-certs") pod "kube-auth-proxy-79b66bc9f5-hpnjw" (UID: "18631460-5a0f-4b5c-b7c8-92e09620ca63") : secret "kube-auth-proxy-tls" not found
Jul 15 19:35:27.113775 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.113749    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18631460-5a0f-4b5c-b7c8-92e09620ca63-tmp\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.119068 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.119042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwrlv\" (UniqueName: \"kubernetes.io/projected/18631460-5a0f-4b5c-b7c8-92e09620ca63-kube-api-access-fwrlv\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.354118 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.354077    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:27.616165 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.616063    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18631460-5a0f-4b5c-b7c8-92e09620ca63-tls-certs\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.618747 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.618717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18631460-5a0f-4b5c-b7c8-92e09620ca63-tls-certs\") pod \"kube-auth-proxy-79b66bc9f5-hpnjw\" (UID: \"18631460-5a0f-4b5c-b7c8-92e09620ca63\") " pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.805739 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.805700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"
Jul 15 19:35:27.950349 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:27.948324    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw"]
Jul 15 19:35:27.952038 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:35:27.952007    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18631460_5a0f_4b5c_b7c8_92e09620ca63.slice/crio-70ce7b52c7ebe9e36394bcc8dae56010af13e4df9758d34262a44e6577cb56f9 WatchSource:0}: Error finding container 70ce7b52c7ebe9e36394bcc8dae56010af13e4df9758d34262a44e6577cb56f9: Status 404 returned error can't find the container with id 70ce7b52c7ebe9e36394bcc8dae56010af13e4df9758d34262a44e6577cb56f9
Jul 15 19:35:28.004330 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:28.004296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:28.429035 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:28.428998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw" event={"ID":"18631460-5a0f-4b5c-b7c8-92e09620ca63","Type":"ContainerStarted","Data":"70ce7b52c7ebe9e36394bcc8dae56010af13e4df9758d34262a44e6577cb56f9"}
Jul 15 19:35:28.649686 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:28.649650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:29.425658 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:29.425617    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:30.924657 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:30.924636    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 19:35:31.445677 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:31.445644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw" event={"ID":"18631460-5a0f-4b5c-b7c8-92e09620ca63","Type":"ContainerStarted","Data":"c11d903b30479b040d52672ce3c9d9cb3043d814e31062b8ca6824d2be6e15e6"}
Jul 15 19:35:31.481671 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:31.481615    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79b66bc9f5-hpnjw" podStartSLOduration=2.513266226 podStartE2EDuration="5.481597453s" totalImagesPullingTime="2.968331227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:31.480267966 +0000 UTC m=+421.558060764" watchObservedRunningTime="2026-07-15 19:35:31.481597453 +0000 UTC m=+421.559390253"
Jul 15 19:35:32.032548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:32.032499    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:32.724259 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:32.724206    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:32.781547 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:32.781499    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:33.129862 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:33.129773    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:33.233365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:33.233331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:33.401099 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:33.401003    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:33.505807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:33.505775    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:33.969026 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:33.968984    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:34.208181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:34.208139    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:34.322681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:34.322596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:35.415837 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:35.415809    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7784c86d4c-ffnll"
Jul 15 19:35:54.067843 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:35:54.067745    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:07.845134 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:07.845090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:07.900840 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:07.900805    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:09.797382 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:09.797344    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:09.848203 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:09.848164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:14.576108 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:14.576075    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:14.630325 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:14.630290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:22.501923 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:22.501890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:22.597032 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:22.596977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:22.666682 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:22.666650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:37.311362 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.311332    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"]
Jul 15 19:36:37.314315 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.314297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.316480 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.316447    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 19:36:37.316480 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.316455    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 19:36:37.317081 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.317060    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dm48d\""
Jul 15 19:36:37.321472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.321388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"]
Jul 15 19:36:37.427738 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.427700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.427935 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.427765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg97l\" (UniqueName: \"kubernetes.io/projected/1cce0045-1991-47dd-83ca-3bcde525eb79-kube-api-access-sg97l\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.427935 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.427857    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.529177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.529133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.529402 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.529195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.529402 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.529296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg97l\" (UniqueName: \"kubernetes.io/projected/1cce0045-1991-47dd-83ca-3bcde525eb79-kube-api-access-sg97l\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.529581 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.529556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.529625 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.529576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.536987 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.536965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg97l\" (UniqueName: \"kubernetes.io/projected/1cce0045-1991-47dd-83ca-3bcde525eb79-kube-api-access-sg97l\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.623800 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.623712    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:37.761292 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.761262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"]
Jul 15 19:36:37.762775 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:36:37.762745    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cce0045_1991_47dd_83ca_3bcde525eb79.slice/crio-54f85dec296c07a87d00985cc01fe02b5281c8b2338cdc805104ff61a6048574 WatchSource:0}: Error finding container 54f85dec296c07a87d00985cc01fe02b5281c8b2338cdc805104ff61a6048574: Status 404 returned error can't find the container with id 54f85dec296c07a87d00985cc01fe02b5281c8b2338cdc805104ff61a6048574
Jul 15 19:36:37.908981 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.908898    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"]
Jul 15 19:36:37.911508 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.911491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:37.921492 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:37.921470    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"]
Jul 15 19:36:38.032737 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.032689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkwx4\" (UniqueName: \"kubernetes.io/projected/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-kube-api-access-mkwx4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.032944 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.032769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.032944 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.032807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.134331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.134279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkwx4\" (UniqueName: \"kubernetes.io/projected/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-kube-api-access-mkwx4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.134510 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.134381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.134510 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.134420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.134853 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.134835    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.134891 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.134871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.142890 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.142856    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkwx4\" (UniqueName: \"kubernetes.io/projected/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-kube-api-access-mkwx4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.221841 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.221802    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:38.356683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.356656    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"]
Jul 15 19:36:38.358243 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:36:38.358202    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfd402c_ee4e_4438_8b61_12d00cfeee2e.slice/crio-63ff03920e27b2765c4873b0a4f8c344bb20b31218e34c97bf00148d8b4dcc37 WatchSource:0}: Error finding container 63ff03920e27b2765c4873b0a4f8c344bb20b31218e34c97bf00148d8b4dcc37: Status 404 returned error can't find the container with id 63ff03920e27b2765c4873b0a4f8c344bb20b31218e34c97bf00148d8b4dcc37
Jul 15 19:36:38.508923 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.508844    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"]
Jul 15 19:36:38.511621 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.511603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.519632 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.519604    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"]
Jul 15 19:36:38.638636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.638590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.638810 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.638675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.638810 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.638704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss5sw\" (UniqueName: \"kubernetes.io/projected/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-kube-api-access-ss5sw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.740051 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.740019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss5sw\" (UniqueName: \"kubernetes.io/projected/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-kube-api-access-ss5sw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.740231 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.740088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.740231 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.740188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.740532 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.740509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.740609 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.740574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.747623 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.747599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss5sw\" (UniqueName: \"kubernetes.io/projected/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-kube-api-access-ss5sw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.766162 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.766079    2564 generic.go:356] "Generic (PLEG): container finished" podID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerID="3fcbd8f0cf05b91708b176e01848e71c0a69abe77c2411011301a626c1ef4968" exitCode=0
Jul 15 19:36:38.766304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.766165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g" event={"ID":"cdfd402c-ee4e-4438-8b61-12d00cfeee2e","Type":"ContainerDied","Data":"3fcbd8f0cf05b91708b176e01848e71c0a69abe77c2411011301a626c1ef4968"}
Jul 15 19:36:38.766304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.766234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g" event={"ID":"cdfd402c-ee4e-4438-8b61-12d00cfeee2e","Type":"ContainerStarted","Data":"63ff03920e27b2765c4873b0a4f8c344bb20b31218e34c97bf00148d8b4dcc37"}
Jul 15 19:36:38.768010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.767983    2564 generic.go:356] "Generic (PLEG): container finished" podID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerID="e709e73dc3d0dc349374f502350954178c6e06f39ef9d5b1f63822b4ef24318f" exitCode=0
Jul 15 19:36:38.768133 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.768046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb" event={"ID":"1cce0045-1991-47dd-83ca-3bcde525eb79","Type":"ContainerDied","Data":"e709e73dc3d0dc349374f502350954178c6e06f39ef9d5b1f63822b4ef24318f"}
Jul 15 19:36:38.768133 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.768083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb" event={"ID":"1cce0045-1991-47dd-83ca-3bcde525eb79","Type":"ContainerStarted","Data":"54f85dec296c07a87d00985cc01fe02b5281c8b2338cdc805104ff61a6048574"}
Jul 15 19:36:38.831473 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.831433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:38.965365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:38.965336    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"]
Jul 15 19:36:38.967267 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:36:38.967214    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4266d3_d64c_4aa7_8862_ec2f2f078be3.slice/crio-fd44170d2834970f1bb6221cc246207c7ac1c66bf7646b3c77950f42736852ac WatchSource:0}: Error finding container fd44170d2834970f1bb6221cc246207c7ac1c66bf7646b3c77950f42736852ac: Status 404 returned error can't find the container with id fd44170d2834970f1bb6221cc246207c7ac1c66bf7646b3c77950f42736852ac
Jul 15 19:36:39.113001 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.112963    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"]
Jul 15 19:36:39.117331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.117289    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.124275 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.124243    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"]
Jul 15 19:36:39.245117 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.245087    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.245341 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.245130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.245341 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.245278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vlf\" (UniqueName: \"kubernetes.io/projected/45ea731d-620a-4baf-ab42-e55e205c6d2f-kube-api-access-49vlf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.346252 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.346116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.346252 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.346180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.346252 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.346254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49vlf\" (UniqueName: \"kubernetes.io/projected/45ea731d-620a-4baf-ab42-e55e205c6d2f-kube-api-access-49vlf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.346622 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.346597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.346686 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.346620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.353782 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.353755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vlf\" (UniqueName: \"kubernetes.io/projected/45ea731d-620a-4baf-ab42-e55e205c6d2f-kube-api-access-49vlf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.434930 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.434892    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:39.641331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.641303    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"]
Jul 15 19:36:39.643104 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:36:39.643073    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ea731d_620a_4baf_ab42_e55e205c6d2f.slice/crio-347f6c2fd798d5da084841efb77a7d7379cfe4dd04874748234a9194dbecc4cd WatchSource:0}: Error finding container 347f6c2fd798d5da084841efb77a7d7379cfe4dd04874748234a9194dbecc4cd: Status 404 returned error can't find the container with id 347f6c2fd798d5da084841efb77a7d7379cfe4dd04874748234a9194dbecc4cd
Jul 15 19:36:39.777972 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.777930    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerID="873f9089596045ce89a1207dbf1b48548407937de1bcec94f9c95818e048936b" exitCode=0
Jul 15 19:36:39.778151 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.778018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d" event={"ID":"dd4266d3-d64c-4aa7-8862-ec2f2f078be3","Type":"ContainerDied","Data":"873f9089596045ce89a1207dbf1b48548407937de1bcec94f9c95818e048936b"}
Jul 15 19:36:39.778151 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.778067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d" event={"ID":"dd4266d3-d64c-4aa7-8862-ec2f2f078be3","Type":"ContainerStarted","Data":"fd44170d2834970f1bb6221cc246207c7ac1c66bf7646b3c77950f42736852ac"}
Jul 15 19:36:39.780399 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.780367    2564 generic.go:356] "Generic (PLEG): container finished" podID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerID="66ef528e59994f9d5e3510b4822d720d1b769410ea038f88d0d9998b1f513cc2" exitCode=0
Jul 15 19:36:39.780399 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.780394    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb" event={"ID":"1cce0045-1991-47dd-83ca-3bcde525eb79","Type":"ContainerDied","Data":"66ef528e59994f9d5e3510b4822d720d1b769410ea038f88d0d9998b1f513cc2"}
Jul 15 19:36:39.782172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.782150    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf" event={"ID":"45ea731d-620a-4baf-ab42-e55e205c6d2f","Type":"ContainerStarted","Data":"242c36c422297be89ff9ef7dd2a564533cb77d5fa3f649ce5f33441b24720d77"}
Jul 15 19:36:39.782303 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:39.782195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf" event={"ID":"45ea731d-620a-4baf-ab42-e55e205c6d2f","Type":"ContainerStarted","Data":"347f6c2fd798d5da084841efb77a7d7379cfe4dd04874748234a9194dbecc4cd"}
Jul 15 19:36:40.788337 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:40.788241    2564 generic.go:356] "Generic (PLEG): container finished" podID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerID="a0779ca2fe5f1078b1e50ea509deaa149dae7e5434907bf6d8eebc39637077f3" exitCode=0
Jul 15 19:36:40.788337 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:40.788322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb" event={"ID":"1cce0045-1991-47dd-83ca-3bcde525eb79","Type":"ContainerDied","Data":"a0779ca2fe5f1078b1e50ea509deaa149dae7e5434907bf6d8eebc39637077f3"}
Jul 15 19:36:40.789749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:40.789724    2564 generic.go:356] "Generic (PLEG): container finished" podID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerID="242c36c422297be89ff9ef7dd2a564533cb77d5fa3f649ce5f33441b24720d77" exitCode=0
Jul 15 19:36:40.789865 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:40.789814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf" event={"ID":"45ea731d-620a-4baf-ab42-e55e205c6d2f","Type":"ContainerDied","Data":"242c36c422297be89ff9ef7dd2a564533cb77d5fa3f649ce5f33441b24720d77"}
Jul 15 19:36:40.791885 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:40.791861    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerID="0dbb47878c2423972c1e784b24f87315d5cfdb69c5cde996cb96f1869819f099" exitCode=0
Jul 15 19:36:40.791970 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:40.791911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d" event={"ID":"dd4266d3-d64c-4aa7-8862-ec2f2f078be3","Type":"ContainerDied","Data":"0dbb47878c2423972c1e784b24f87315d5cfdb69c5cde996cb96f1869819f099"}
Jul 15 19:36:41.799070 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.799034    2564 generic.go:356] "Generic (PLEG): container finished" podID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerID="7b80691e6524c518aac79026a3046d4e65df5f0366ecf3dc1263c65c183596f7" exitCode=0
Jul 15 19:36:41.799558 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.799137    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf" event={"ID":"45ea731d-620a-4baf-ab42-e55e205c6d2f","Type":"ContainerDied","Data":"7b80691e6524c518aac79026a3046d4e65df5f0366ecf3dc1263c65c183596f7"}
Jul 15 19:36:41.801097 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.801068    2564 generic.go:356] "Generic (PLEG): container finished" podID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerID="a2358b3e9f0c9d747846b3086c7f4d1c184b855d4d95bf7c2756ccf950881e3d" exitCode=0
Jul 15 19:36:41.801183 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.801144    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g" event={"ID":"cdfd402c-ee4e-4438-8b61-12d00cfeee2e","Type":"ContainerDied","Data":"a2358b3e9f0c9d747846b3086c7f4d1c184b855d4d95bf7c2756ccf950881e3d"}
Jul 15 19:36:41.803406 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.803386    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerID="9eade5eae7078b094350cf8c6d98405e6486d2898f27aeb58313f7a5fa926c96" exitCode=0
Jul 15 19:36:41.803487 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.803445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d" event={"ID":"dd4266d3-d64c-4aa7-8862-ec2f2f078be3","Type":"ContainerDied","Data":"9eade5eae7078b094350cf8c6d98405e6486d2898f27aeb58313f7a5fa926c96"}
Jul 15 19:36:41.964101 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:41.964077    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:42.073280 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.073187    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-bundle\") pod \"1cce0045-1991-47dd-83ca-3bcde525eb79\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") "
Jul 15 19:36:42.073425 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.073313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-util\") pod \"1cce0045-1991-47dd-83ca-3bcde525eb79\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") "
Jul 15 19:36:42.073425 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.073335    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg97l\" (UniqueName: \"kubernetes.io/projected/1cce0045-1991-47dd-83ca-3bcde525eb79-kube-api-access-sg97l\") pod \"1cce0045-1991-47dd-83ca-3bcde525eb79\" (UID: \"1cce0045-1991-47dd-83ca-3bcde525eb79\") "
Jul 15 19:36:42.073726 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.073699    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-bundle" (OuterVolumeSpecName: "bundle") pod "1cce0045-1991-47dd-83ca-3bcde525eb79" (UID: "1cce0045-1991-47dd-83ca-3bcde525eb79"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:42.075828 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.075804    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cce0045-1991-47dd-83ca-3bcde525eb79-kube-api-access-sg97l" (OuterVolumeSpecName: "kube-api-access-sg97l") pod "1cce0045-1991-47dd-83ca-3bcde525eb79" (UID: "1cce0045-1991-47dd-83ca-3bcde525eb79"). InnerVolumeSpecName "kube-api-access-sg97l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:36:42.078785 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.078751    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-util" (OuterVolumeSpecName: "util") pod "1cce0045-1991-47dd-83ca-3bcde525eb79" (UID: "1cce0045-1991-47dd-83ca-3bcde525eb79"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:42.174558 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.174516    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:42.174558 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.174550    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg97l\" (UniqueName: \"kubernetes.io/projected/1cce0045-1991-47dd-83ca-3bcde525eb79-kube-api-access-sg97l\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:42.174558 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.174563    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cce0045-1991-47dd-83ca-3bcde525eb79-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:42.810741 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.810664    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb"
Jul 15 19:36:42.811210 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.810654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb" event={"ID":"1cce0045-1991-47dd-83ca-3bcde525eb79","Type":"ContainerDied","Data":"54f85dec296c07a87d00985cc01fe02b5281c8b2338cdc805104ff61a6048574"}
Jul 15 19:36:42.811210 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.810777    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54f85dec296c07a87d00985cc01fe02b5281c8b2338cdc805104ff61a6048574"
Jul 15 19:36:42.813120 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.813093    2564 generic.go:356] "Generic (PLEG): container finished" podID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerID="76fb62517b7db860e2044c77441132fcca62a7489346fdae98cb2398f7f5edbf" exitCode=0
Jul 15 19:36:42.813258 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.813124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf" event={"ID":"45ea731d-620a-4baf-ab42-e55e205c6d2f","Type":"ContainerDied","Data":"76fb62517b7db860e2044c77441132fcca62a7489346fdae98cb2398f7f5edbf"}
Jul 15 19:36:42.815115 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.815092    2564 generic.go:356] "Generic (PLEG): container finished" podID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerID="4597a570ac3b320ca8fbc2a1d1bf87fcd3f0390686af3bdac6a603329bde6884" exitCode=0
Jul 15 19:36:42.815215 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.815176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g" event={"ID":"cdfd402c-ee4e-4438-8b61-12d00cfeee2e","Type":"ContainerDied","Data":"4597a570ac3b320ca8fbc2a1d1bf87fcd3f0390686af3bdac6a603329bde6884"}
Jul 15 19:36:42.946332 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:42.946308    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:43.085155 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.085066    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-util\") pod \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") "
Jul 15 19:36:43.085155 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.085129    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss5sw\" (UniqueName: \"kubernetes.io/projected/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-kube-api-access-ss5sw\") pod \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") "
Jul 15 19:36:43.085375 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.085173    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-bundle\") pod \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\" (UID: \"dd4266d3-d64c-4aa7-8862-ec2f2f078be3\") "
Jul 15 19:36:43.085824 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.085791    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-bundle" (OuterVolumeSpecName: "bundle") pod "dd4266d3-d64c-4aa7-8862-ec2f2f078be3" (UID: "dd4266d3-d64c-4aa7-8862-ec2f2f078be3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:43.087445 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.087412    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-kube-api-access-ss5sw" (OuterVolumeSpecName: "kube-api-access-ss5sw") pod "dd4266d3-d64c-4aa7-8862-ec2f2f078be3" (UID: "dd4266d3-d64c-4aa7-8862-ec2f2f078be3"). InnerVolumeSpecName "kube-api-access-ss5sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:36:43.090409 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.090389    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-util" (OuterVolumeSpecName: "util") pod "dd4266d3-d64c-4aa7-8862-ec2f2f078be3" (UID: "dd4266d3-d64c-4aa7-8862-ec2f2f078be3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:43.186699 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.186655    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:43.186699 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.186691    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss5sw\" (UniqueName: \"kubernetes.io/projected/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-kube-api-access-ss5sw\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:43.186699 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.186704    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd4266d3-d64c-4aa7-8862-ec2f2f078be3-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:43.821689 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.821659    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d"
Jul 15 19:36:43.821689 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.821683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d" event={"ID":"dd4266d3-d64c-4aa7-8862-ec2f2f078be3","Type":"ContainerDied","Data":"fd44170d2834970f1bb6221cc246207c7ac1c66bf7646b3c77950f42736852ac"}
Jul 15 19:36:43.822153 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.821734    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd44170d2834970f1bb6221cc246207c7ac1c66bf7646b3c77950f42736852ac"
Jul 15 19:36:43.982297 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.982275    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:43.985106 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:43.985087    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:44.095208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095111    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-bundle\") pod \"45ea731d-620a-4baf-ab42-e55e205c6d2f\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") "
Jul 15 19:36:44.095208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095156    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-util\") pod \"45ea731d-620a-4baf-ab42-e55e205c6d2f\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") "
Jul 15 19:36:44.095208 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095193    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-util\") pod \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") "
Jul 15 19:36:44.095540 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095277    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49vlf\" (UniqueName: \"kubernetes.io/projected/45ea731d-620a-4baf-ab42-e55e205c6d2f-kube-api-access-49vlf\") pod \"45ea731d-620a-4baf-ab42-e55e205c6d2f\" (UID: \"45ea731d-620a-4baf-ab42-e55e205c6d2f\") "
Jul 15 19:36:44.095540 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095338    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-bundle\") pod \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") "
Jul 15 19:36:44.095540 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095362    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkwx4\" (UniqueName: \"kubernetes.io/projected/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-kube-api-access-mkwx4\") pod \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\" (UID: \"cdfd402c-ee4e-4438-8b61-12d00cfeee2e\") "
Jul 15 19:36:44.095817 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-bundle" (OuterVolumeSpecName: "bundle") pod "45ea731d-620a-4baf-ab42-e55e205c6d2f" (UID: "45ea731d-620a-4baf-ab42-e55e205c6d2f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:44.096005 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.095980    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-bundle" (OuterVolumeSpecName: "bundle") pod "cdfd402c-ee4e-4438-8b61-12d00cfeee2e" (UID: "cdfd402c-ee4e-4438-8b61-12d00cfeee2e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:44.097671 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.097644    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-kube-api-access-mkwx4" (OuterVolumeSpecName: "kube-api-access-mkwx4") pod "cdfd402c-ee4e-4438-8b61-12d00cfeee2e" (UID: "cdfd402c-ee4e-4438-8b61-12d00cfeee2e"). InnerVolumeSpecName "kube-api-access-mkwx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:36:44.098131 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.098112    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45ea731d-620a-4baf-ab42-e55e205c6d2f-kube-api-access-49vlf" (OuterVolumeSpecName: "kube-api-access-49vlf") pod "45ea731d-620a-4baf-ab42-e55e205c6d2f" (UID: "45ea731d-620a-4baf-ab42-e55e205c6d2f"). InnerVolumeSpecName "kube-api-access-49vlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:36:44.101401 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.101359    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-util" (OuterVolumeSpecName: "util") pod "cdfd402c-ee4e-4438-8b61-12d00cfeee2e" (UID: "cdfd402c-ee4e-4438-8b61-12d00cfeee2e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:44.101743 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.101722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-util" (OuterVolumeSpecName: "util") pod "45ea731d-620a-4baf-ab42-e55e205c6d2f" (UID: "45ea731d-620a-4baf-ab42-e55e205c6d2f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:36:44.196412 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.196379    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49vlf\" (UniqueName: \"kubernetes.io/projected/45ea731d-620a-4baf-ab42-e55e205c6d2f-kube-api-access-49vlf\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:44.196412 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.196406    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:44.196412 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.196415    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkwx4\" (UniqueName: \"kubernetes.io/projected/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-kube-api-access-mkwx4\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:44.196645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.196424    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:44.196645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.196433    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45ea731d-620a-4baf-ab42-e55e205c6d2f-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:44.196645 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.196441    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdfd402c-ee4e-4438-8b61-12d00cfeee2e-util\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:36:44.828797 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.828711    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g"
Jul 15 19:36:44.828797 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.828726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g" event={"ID":"cdfd402c-ee4e-4438-8b61-12d00cfeee2e","Type":"ContainerDied","Data":"63ff03920e27b2765c4873b0a4f8c344bb20b31218e34c97bf00148d8b4dcc37"}
Jul 15 19:36:44.828797 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.828777    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63ff03920e27b2765c4873b0a4f8c344bb20b31218e34c97bf00148d8b4dcc37"
Jul 15 19:36:44.830824 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.830792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf" event={"ID":"45ea731d-620a-4baf-ab42-e55e205c6d2f","Type":"ContainerDied","Data":"347f6c2fd798d5da084841efb77a7d7379cfe4dd04874748234a9194dbecc4cd"}
Jul 15 19:36:44.830974 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.830840    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="347f6c2fd798d5da084841efb77a7d7379cfe4dd04874748234a9194dbecc4cd"
Jul 15 19:36:44.830974 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:44.830854    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf"
Jul 15 19:36:49.731370 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:49.731334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:36:53.929375 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:36:53.929342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:00.989093 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:00.989047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:01.877732 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.877694    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-556b9c656b-hf7qp"]
Jul 15 19:37:01.878089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878077    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="pull"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878091    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="pull"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878099    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="util"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878105    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="util"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878113    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="extract"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878118    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="extract"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878125    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="extract"
Jul 15 19:37:01.878129 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878130    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878138    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="pull"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878144    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="pull"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878155    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="util"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878161    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="util"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878166    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="util"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878170    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="util"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878176    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="pull"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878181    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="pull"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878187    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="pull"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878191    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="pull"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878205    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878210    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878215    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878233    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878239    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="util"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878244    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="util"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878305    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdfd402c-ee4e-4438-8b61-12d00cfeee2e" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878312    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd4266d3-d64c-4aa7-8862-ec2f2f078be3" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878320    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="45ea731d-620a-4baf-ab42-e55e205c6d2f" containerName="extract"
Jul 15 19:37:01.878359 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.878327    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cce0045-1991-47dd-83ca-3bcde525eb79" containerName="extract"
Jul 15 19:37:01.881274 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.881257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.893662 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.893634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556b9c656b-hf7qp"]
Jul 15 19:37:01.942813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942773    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-trusted-ca-bundle\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.942813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-config\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.943058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcc6f\" (UniqueName: \"kubernetes.io/projected/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-kube-api-access-xcc6f\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.943058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942869    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-serving-cert\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.943058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942894    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-service-ca\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.943058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-oauth-config\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.943058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.942944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-oauth-serving-cert\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:01.964177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:01.964140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:02.044027 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.043970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-config\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044027 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcc6f\" (UniqueName: \"kubernetes.io/projected/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-kube-api-access-xcc6f\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-serving-cert\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044077    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-service-ca\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-oauth-config\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-oauth-serving-cert\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-trusted-ca-bundle\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044829    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-config\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.044900 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-service-ca\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.045017 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.044920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-oauth-serving-cert\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.045256 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.045204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-trusted-ca-bundle\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.047441 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.047413    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-oauth-config\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.047537 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.047490    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-console-serving-cert\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.051210 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.051190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcc6f\" (UniqueName: \"kubernetes.io/projected/f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad-kube-api-access-xcc6f\") pod \"console-556b9c656b-hf7qp\" (UID: \"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad\") " pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.191947 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.191854    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:02.328003 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.327976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556b9c656b-hf7qp"]
Jul 15 19:37:02.329825 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:37:02.329798    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dbbd7e_e131_45e0_9bf4_8c01a08f36ad.slice/crio-d1731b7623bcbf3ee7197b8ceddabbf9998c4fb3283b5bc4b5a80567cf3872da WatchSource:0}: Error finding container d1731b7623bcbf3ee7197b8ceddabbf9998c4fb3283b5bc4b5a80567cf3872da: Status 404 returned error can't find the container with id d1731b7623bcbf3ee7197b8ceddabbf9998c4fb3283b5bc4b5a80567cf3872da
Jul 15 19:37:02.927542 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.927503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556b9c656b-hf7qp" event={"ID":"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad","Type":"ContainerStarted","Data":"bab014646618490ffda67975832cec20de0c33b0748871a5354995bdbe8b5c10"}
Jul 15 19:37:02.927756 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.927563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556b9c656b-hf7qp" event={"ID":"f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad","Type":"ContainerStarted","Data":"d1731b7623bcbf3ee7197b8ceddabbf9998c4fb3283b5bc4b5a80567cf3872da"}
Jul 15 19:37:02.945110 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:02.945053    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-556b9c656b-hf7qp" podStartSLOduration=1.945036145 podStartE2EDuration="1.945036145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:37:02.944282613 +0000 UTC m=+513.022075414" watchObservedRunningTime="2026-07-15 19:37:02.945036145 +0000 UTC m=+513.022828945"
Jul 15 19:37:04.133701 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:04.133654    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:04.936601 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:04.936554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:07.967650 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:07.967609    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:08.544902 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:08.544854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:12.192497 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:12.192454    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:12.192967 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:12.192560    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:12.197450 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:12.197426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:12.981058 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:12.981026    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-556b9c656b-hf7qp"
Jul 15 19:37:13.024845 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:13.024796    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84795c6465-srd5s"]
Jul 15 19:37:13.717794 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:13.717757    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:13.778276 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:13.778243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:22.174919 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:22.174834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:23.976892 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:23.976852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:25.397011 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:25.396976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:26.773314 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:26.773281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:28.560981 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:28.560942    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:28.616987 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:28.616950    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:28.740658 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:28.740624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:28.795750 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:28.795711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:35.541665 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:35.541621    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:35.657524 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:35.657486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:38.052740 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.052674    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84795c6465-srd5s" podUID="3286494f-1767-4ce2-995b-9d586704d03e" containerName="console" containerID="cri-o://e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff" gracePeriod=15
Jul 15 19:37:38.297315 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.297290    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:37:38.375345 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375262    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-serving-cert\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375345 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375346    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-console-config\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375567 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375363    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-trusted-ca-bundle\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375567 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bf2c\" (UniqueName: \"kubernetes.io/projected/3286494f-1767-4ce2-995b-9d586704d03e-kube-api-access-8bf2c\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375567 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375408    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-oauth-serving-cert\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375567 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375430    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-service-ca\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375567 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375467    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-oauth-config\") pod \"3286494f-1767-4ce2-995b-9d586704d03e\" (UID: \"3286494f-1767-4ce2-995b-9d586704d03e\") "
Jul 15 19:37:38.375827 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375807    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-console-config" (OuterVolumeSpecName: "console-config") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:37:38.375886 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375840    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:37:38.375886 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375857    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-service-ca" (OuterVolumeSpecName: "service-ca") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:37:38.375960 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.375875    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:37:38.377629 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.377605    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:37:38.377744 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.377658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3286494f-1767-4ce2-995b-9d586704d03e-kube-api-access-8bf2c" (OuterVolumeSpecName: "kube-api-access-8bf2c") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "kube-api-access-8bf2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:37:38.377807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.377772    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3286494f-1767-4ce2-995b-9d586704d03e" (UID: "3286494f-1767-4ce2-995b-9d586704d03e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:37:38.476777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476736    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.476777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476767    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.476777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476778    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.476777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476787    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bf2c\" (UniqueName: \"kubernetes.io/projected/3286494f-1767-4ce2-995b-9d586704d03e-kube-api-access-8bf2c\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.477041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476797    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.477041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476805    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3286494f-1767-4ce2-995b-9d586704d03e-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.477041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.476813    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3286494f-1767-4ce2-995b-9d586704d03e-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:37:38.550216 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.550181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:38.650563 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:38.650446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:39.116548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.116516    2564 generic.go:356] "Generic (PLEG): container finished" podID="3286494f-1767-4ce2-995b-9d586704d03e" containerID="e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff" exitCode=0
Jul 15 19:37:39.116994 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.116559    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84795c6465-srd5s"
Jul 15 19:37:39.116994 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.116565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84795c6465-srd5s" event={"ID":"3286494f-1767-4ce2-995b-9d586704d03e","Type":"ContainerDied","Data":"e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff"}
Jul 15 19:37:39.116994 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.116606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84795c6465-srd5s" event={"ID":"3286494f-1767-4ce2-995b-9d586704d03e","Type":"ContainerDied","Data":"070bcff8243ea6de835924e0902927339c81b9a6f297e91a6e1a1b714571c5ad"}
Jul 15 19:37:39.116994 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.116625    2564 scope.go:117] "RemoveContainer" containerID="e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff"
Jul 15 19:37:39.125631 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.125609    2564 scope.go:117] "RemoveContainer" containerID="e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff"
Jul 15 19:37:39.125919 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:37:39.125899    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff\": container with ID starting with e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff not found: ID does not exist" containerID="e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff"
Jul 15 19:37:39.125966 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.125935    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff"} err="failed to get container status \"e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff\": rpc error: code = NotFound desc = could not find container \"e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff\": container with ID starting with e3e5fac72a1147de71ead7955fb8dca06eb5f002f0d34b94dedfe902b0f484ff not found: ID does not exist"
Jul 15 19:37:39.150939 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.150898    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84795c6465-srd5s"]
Jul 15 19:37:39.155029 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:39.155002    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84795c6465-srd5s"]
Jul 15 19:37:40.499602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:40.499565    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3286494f-1767-4ce2-995b-9d586704d03e" path="/var/lib/kubelet/pods/3286494f-1767-4ce2-995b-9d586704d03e/volumes"
Jul 15 19:37:45.060761 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:45.060717    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:45.163171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:45.163134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:51.544652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:51.544616    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:51.644695 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:51.644658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:54.162911 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:54.162872    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:54.260747 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:54.260698    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:57.749667 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:57.749623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:57.814106 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:57.814062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:58.332852 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:58.332815    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:58.488544 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:58.488509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:37:58.648743 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:37:58.648625    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:05.385644 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:05.385604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:05.485889 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:05.485841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:11.347527 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:11.347474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:11.475844 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:11.475804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:13.431015 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.430968    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:13.437120 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.437093    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:13.437482 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.437469    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3286494f-1767-4ce2-995b-9d586704d03e" containerName="console"
Jul 15 19:38:13.437528 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.437484    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3286494f-1767-4ce2-995b-9d586704d03e" containerName="console"
Jul 15 19:38:13.437562 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.437552    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3286494f-1767-4ce2-995b-9d586704d03e" containerName="console"
Jul 15 19:38:13.440596 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.440580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.442933 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.442913    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 19:38:13.443070 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.442965    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 19:38:13.443070 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.443042    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 19:38:13.443652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.443631    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dm48d\""
Jul 15 19:38:13.450241 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.450192    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:13.486320 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.486280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2304eb77-4df2-4b7c-a896-399a6baf7dad-config-file\") pod \"limitador-limitador-64c94fd64f-swk7c\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.486495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.486368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q96m5\" (UniqueName: \"kubernetes.io/projected/2304eb77-4df2-4b7c-a896-399a6baf7dad-kube-api-access-q96m5\") pod \"limitador-limitador-64c94fd64f-swk7c\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.542948 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.542917    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:13.548269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.548208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:13.587430 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.587399    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q96m5\" (UniqueName: \"kubernetes.io/projected/2304eb77-4df2-4b7c-a896-399a6baf7dad-kube-api-access-q96m5\") pod \"limitador-limitador-64c94fd64f-swk7c\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.587591 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.587462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2304eb77-4df2-4b7c-a896-399a6baf7dad-config-file\") pod \"limitador-limitador-64c94fd64f-swk7c\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.588092 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.588073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2304eb77-4df2-4b7c-a896-399a6baf7dad-config-file\") pod \"limitador-limitador-64c94fd64f-swk7c\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.594886 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.594860    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q96m5\" (UniqueName: \"kubernetes.io/projected/2304eb77-4df2-4b7c-a896-399a6baf7dad-kube-api-access-q96m5\") pod \"limitador-limitador-64c94fd64f-swk7c\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.649850 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.649809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:13.752170 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.752136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:13.889301 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:13.889270    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:13.891204 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:13.891178    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2304eb77_4df2_4b7c_a896_399a6baf7dad.slice/crio-11bbd3214f5d53d077f405247adaf6f7843897c1f5956670615833ebe97914b4 WatchSource:0}: Error finding container 11bbd3214f5d53d077f405247adaf6f7843897c1f5956670615833ebe97914b4: Status 404 returned error can't find the container with id 11bbd3214f5d53d077f405247adaf6f7843897c1f5956670615833ebe97914b4
Jul 15 19:38:14.255357 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.255325    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-nskjf"]
Jul 15 19:38:14.260532 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.260491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:14.264681 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.264651    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ssvf4\""
Jul 15 19:38:14.267634 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.267593    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nskjf"]
Jul 15 19:38:14.279021 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.278926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:14.293661 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.293629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrv5h\" (UniqueName: \"kubernetes.io/projected/7b68cd5c-95fc-4cef-80b8-285b2a8f443c-kube-api-access-rrv5h\") pod \"authorino-58cd955675-nskjf\" (UID: \"7b68cd5c-95fc-4cef-80b8-285b2a8f443c\") " pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:14.299020 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.298948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c" event={"ID":"2304eb77-4df2-4b7c-a896-399a6baf7dad","Type":"ContainerStarted","Data":"11bbd3214f5d53d077f405247adaf6f7843897c1f5956670615833ebe97914b4"}
Jul 15 19:38:14.340770 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.340726    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:14.394363 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.394323    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrv5h\" (UniqueName: \"kubernetes.io/projected/7b68cd5c-95fc-4cef-80b8-285b2a8f443c-kube-api-access-rrv5h\") pod \"authorino-58cd955675-nskjf\" (UID: \"7b68cd5c-95fc-4cef-80b8-285b2a8f443c\") " pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:14.401966 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.401941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrv5h\" (UniqueName: \"kubernetes.io/projected/7b68cd5c-95fc-4cef-80b8-285b2a8f443c-kube-api-access-rrv5h\") pod \"authorino-58cd955675-nskjf\" (UID: \"7b68cd5c-95fc-4cef-80b8-285b2a8f443c\") " pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:14.575631 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.575544    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:14.650061 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.650029    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-8cv6q"]
Jul 15 19:38:14.661464 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.655605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:14.663269 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.663196    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8cv6q"]
Jul 15 19:38:14.698004 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.697967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46fkn\" (UniqueName: \"kubernetes.io/projected/4443b365-5ec6-4cef-904f-3e19c277f8f5-kube-api-access-46fkn\") pod \"authorino-75d556bd77-8cv6q\" (UID: \"4443b365-5ec6-4cef-904f-3e19c277f8f5\") " pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:14.774808 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.774352    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nskjf"]
Jul 15 19:38:14.782103 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:14.782074    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b68cd5c_95fc_4cef_80b8_285b2a8f443c.slice/crio-dcb6b32a4e62064b0ab83723849cd73944cb9879494730ffe543c4378538ac56 WatchSource:0}: Error finding container dcb6b32a4e62064b0ab83723849cd73944cb9879494730ffe543c4378538ac56: Status 404 returned error can't find the container with id dcb6b32a4e62064b0ab83723849cd73944cb9879494730ffe543c4378538ac56
Jul 15 19:38:14.799325 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.799276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46fkn\" (UniqueName: \"kubernetes.io/projected/4443b365-5ec6-4cef-904f-3e19c277f8f5-kube-api-access-46fkn\") pod \"authorino-75d556bd77-8cv6q\" (UID: \"4443b365-5ec6-4cef-904f-3e19c277f8f5\") " pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:14.808049 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.808023    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46fkn\" (UniqueName: \"kubernetes.io/projected/4443b365-5ec6-4cef-904f-3e19c277f8f5-kube-api-access-46fkn\") pod \"authorino-75d556bd77-8cv6q\" (UID: \"4443b365-5ec6-4cef-904f-3e19c277f8f5\") " pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:14.977848 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:14.977811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:15.121171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:15.121146    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8cv6q"]
Jul 15 19:38:15.123198 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:15.123168    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4443b365_5ec6_4cef_904f_3e19c277f8f5.slice/crio-578c14bac25f223227b021258981b7c710d5b2d78630db8bfce390e56bfba9b0 WatchSource:0}: Error finding container 578c14bac25f223227b021258981b7c710d5b2d78630db8bfce390e56bfba9b0: Status 404 returned error can't find the container with id 578c14bac25f223227b021258981b7c710d5b2d78630db8bfce390e56bfba9b0
Jul 15 19:38:15.307508 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:15.307421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8cv6q" event={"ID":"4443b365-5ec6-4cef-904f-3e19c277f8f5","Type":"ContainerStarted","Data":"578c14bac25f223227b021258981b7c710d5b2d78630db8bfce390e56bfba9b0"}
Jul 15 19:38:15.308895 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:15.308860    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nskjf" event={"ID":"7b68cd5c-95fc-4cef-80b8-285b2a8f443c","Type":"ContainerStarted","Data":"dcb6b32a4e62064b0ab83723849cd73944cb9879494730ffe543c4378538ac56"}
Jul 15 19:38:19.335976 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.335936    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8cv6q" event={"ID":"4443b365-5ec6-4cef-904f-3e19c277f8f5","Type":"ContainerStarted","Data":"d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7"}
Jul 15 19:38:19.337534 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.337505    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c" event={"ID":"2304eb77-4df2-4b7c-a896-399a6baf7dad","Type":"ContainerStarted","Data":"9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4"}
Jul 15 19:38:19.337687 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.337605    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:19.338929 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.338897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nskjf" event={"ID":"7b68cd5c-95fc-4cef-80b8-285b2a8f443c","Type":"ContainerStarted","Data":"85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97"}
Jul 15 19:38:19.350828 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.350780    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-8cv6q" podStartSLOduration=1.734373904 podStartE2EDuration="5.350767246s" totalImagesPullingTime="3.616393342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:19.348933883 +0000 UTC m=+589.426726681" watchObservedRunningTime="2026-07-15 19:38:19.350767246 +0000 UTC m=+589.428560044"
Jul 15 19:38:19.363490 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.363441    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-nskjf" podStartSLOduration=1.47442454 podStartE2EDuration="5.36342423s" totalImagesPullingTime="3.88899969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:19.361339684 +0000 UTC m=+589.439132691" watchObservedRunningTime="2026-07-15 19:38:19.36342423 +0000 UTC m=+589.441217031"
Jul 15 19:38:19.378636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.378571    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c" podStartSLOduration=1.534900304 podStartE2EDuration="6.378550176s" totalImagesPullingTime="4.843649872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:19.37778133 +0000 UTC m=+589.455574148" watchObservedRunningTime="2026-07-15 19:38:19.378550176 +0000 UTC m=+589.456342977"
Jul 15 19:38:19.393618 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:19.393575    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nskjf"]
Jul 15 19:38:21.349949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:21.349905    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-nskjf" podUID="7b68cd5c-95fc-4cef-80b8-285b2a8f443c" containerName="authorino" containerID="cri-o://85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97" gracePeriod=30
Jul 15 19:38:21.604418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:21.604359    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:21.667613 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:21.667577    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrv5h\" (UniqueName: \"kubernetes.io/projected/7b68cd5c-95fc-4cef-80b8-285b2a8f443c-kube-api-access-rrv5h\") pod \"7b68cd5c-95fc-4cef-80b8-285b2a8f443c\" (UID: \"7b68cd5c-95fc-4cef-80b8-285b2a8f443c\") "
Jul 15 19:38:21.669964 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:21.669933    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b68cd5c-95fc-4cef-80b8-285b2a8f443c-kube-api-access-rrv5h" (OuterVolumeSpecName: "kube-api-access-rrv5h") pod "7b68cd5c-95fc-4cef-80b8-285b2a8f443c" (UID: "7b68cd5c-95fc-4cef-80b8-285b2a8f443c"). InnerVolumeSpecName "kube-api-access-rrv5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:38:21.769234 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:21.769185    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrv5h\" (UniqueName: \"kubernetes.io/projected/7b68cd5c-95fc-4cef-80b8-285b2a8f443c-kube-api-access-rrv5h\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:38:22.356048 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.356016    2564 generic.go:356] "Generic (PLEG): container finished" podID="7b68cd5c-95fc-4cef-80b8-285b2a8f443c" containerID="85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97" exitCode=0
Jul 15 19:38:22.356471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.356061    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nskjf"
Jul 15 19:38:22.356471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.356105    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nskjf" event={"ID":"7b68cd5c-95fc-4cef-80b8-285b2a8f443c","Type":"ContainerDied","Data":"85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97"}
Jul 15 19:38:22.356471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.356142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nskjf" event={"ID":"7b68cd5c-95fc-4cef-80b8-285b2a8f443c","Type":"ContainerDied","Data":"dcb6b32a4e62064b0ab83723849cd73944cb9879494730ffe543c4378538ac56"}
Jul 15 19:38:22.356471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.356163    2564 scope.go:117] "RemoveContainer" containerID="85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97"
Jul 15 19:38:22.368109 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.368083    2564 scope.go:117] "RemoveContainer" containerID="85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97"
Jul 15 19:38:22.368473 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:38:22.368454    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97\": container with ID starting with 85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97 not found: ID does not exist" containerID="85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97"
Jul 15 19:38:22.368619 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.368483    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97"} err="failed to get container status \"85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97\": rpc error: code = NotFound desc = could not find container \"85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97\": container with ID starting with 85b688dc4b16e2f70f98ff86d51854422577d3fd68ffcd787c73697bf3332c97 not found: ID does not exist"
Jul 15 19:38:22.377722 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.377702    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nskjf"]
Jul 15 19:38:22.380908 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.380879    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-nskjf"]
Jul 15 19:38:22.493549 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:22.493518    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b68cd5c-95fc-4cef-80b8-285b2a8f443c" path="/var/lib/kubelet/pods/7b68cd5c-95fc-4cef-80b8-285b2a8f443c/volumes"
Jul 15 19:38:29.145694 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:29.145643    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:29.146198 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:29.145934    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c" podUID="2304eb77-4df2-4b7c-a896-399a6baf7dad" containerName="limitador" containerID="cri-o://9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4" gracePeriod=30
Jul 15 19:38:29.146653 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:29.146634    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:30.092510 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.092485    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:30.249023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.248940    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2304eb77-4df2-4b7c-a896-399a6baf7dad-config-file\") pod \"2304eb77-4df2-4b7c-a896-399a6baf7dad\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") "
Jul 15 19:38:30.249023 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.248989    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q96m5\" (UniqueName: \"kubernetes.io/projected/2304eb77-4df2-4b7c-a896-399a6baf7dad-kube-api-access-q96m5\") pod \"2304eb77-4df2-4b7c-a896-399a6baf7dad\" (UID: \"2304eb77-4df2-4b7c-a896-399a6baf7dad\") "
Jul 15 19:38:30.249433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.249297    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2304eb77-4df2-4b7c-a896-399a6baf7dad-config-file" (OuterVolumeSpecName: "config-file") pod "2304eb77-4df2-4b7c-a896-399a6baf7dad" (UID: "2304eb77-4df2-4b7c-a896-399a6baf7dad"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:38:30.251146 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.251124    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2304eb77-4df2-4b7c-a896-399a6baf7dad-kube-api-access-q96m5" (OuterVolumeSpecName: "kube-api-access-q96m5") pod "2304eb77-4df2-4b7c-a896-399a6baf7dad" (UID: "2304eb77-4df2-4b7c-a896-399a6baf7dad"). InnerVolumeSpecName "kube-api-access-q96m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:38:30.349716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.349676    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2304eb77-4df2-4b7c-a896-399a6baf7dad-config-file\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:38:30.349716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.349708    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q96m5\" (UniqueName: \"kubernetes.io/projected/2304eb77-4df2-4b7c-a896-399a6baf7dad-kube-api-access-q96m5\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:38:30.396974 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.396945    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:38:30.397550 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.397529    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:38:30.400160 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.400126    2564 generic.go:356] "Generic (PLEG): container finished" podID="2304eb77-4df2-4b7c-a896-399a6baf7dad" containerID="9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4" exitCode=0
Jul 15 19:38:30.400304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.400158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c" event={"ID":"2304eb77-4df2-4b7c-a896-399a6baf7dad","Type":"ContainerDied","Data":"9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4"}
Jul 15 19:38:30.400304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.400197    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c"
Jul 15 19:38:30.400304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.400272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-swk7c" event={"ID":"2304eb77-4df2-4b7c-a896-399a6baf7dad","Type":"ContainerDied","Data":"11bbd3214f5d53d077f405247adaf6f7843897c1f5956670615833ebe97914b4"}
Jul 15 19:38:30.400521 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.400310    2564 scope.go:117] "RemoveContainer" containerID="9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4"
Jul 15 19:38:30.409936 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.409918    2564 scope.go:117] "RemoveContainer" containerID="9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4"
Jul 15 19:38:30.410256 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:38:30.410214    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4\": container with ID starting with 9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4 not found: ID does not exist" containerID="9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4"
Jul 15 19:38:30.410360 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.410263    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4"} err="failed to get container status \"9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4\": rpc error: code = NotFound desc = could not find container \"9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4\": container with ID starting with 9fa3287cbcbd0cdfe117fd0b2d93513a1f3e317a863593501a2801c094f1eeb4 not found: ID does not exist"
Jul 15 19:38:30.422168 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.422134    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:30.427012 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.426988    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-swk7c"]
Jul 15 19:38:30.493314 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:30.493286    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2304eb77-4df2-4b7c-a896-399a6baf7dad" path="/var/lib/kubelet/pods/2304eb77-4df2-4b7c-a896-399a6baf7dad/volumes"
Jul 15 19:38:31.651749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:31.651712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:31.747062 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:31.747026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:32.146611 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.146577    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"]
Jul 15 19:38:32.146973 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.146954    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2304eb77-4df2-4b7c-a896-399a6baf7dad" containerName="limitador"
Jul 15 19:38:32.147055 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.146975    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2304eb77-4df2-4b7c-a896-399a6baf7dad" containerName="limitador"
Jul 15 19:38:32.147055 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.147011    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b68cd5c-95fc-4cef-80b8-285b2a8f443c" containerName="authorino"
Jul 15 19:38:32.147055 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.147020    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b68cd5c-95fc-4cef-80b8-285b2a8f443c" containerName="authorino"
Jul 15 19:38:32.147149 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.147109    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b68cd5c-95fc-4cef-80b8-285b2a8f443c" containerName="authorino"
Jul 15 19:38:32.147149 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.147121    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2304eb77-4df2-4b7c-a896-399a6baf7dad" containerName="limitador"
Jul 15 19:38:32.152286 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.152262    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.154559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.154530    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 19:38:32.154705 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.154607    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-nfsmz\""
Jul 15 19:38:32.154705 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.154612    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 19:38:32.155410 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.155389    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 19:38:32.158999 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.158976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"]
Jul 15 19:38:32.184609 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.184578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:32.265891 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.265844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxqlw\" (UniqueName: \"kubernetes.io/projected/254ac940-9cf6-4025-aa45-117c82689fa1-kube-api-access-wxqlw\") pod \"postgres-69fb4594cf-q68xn\" (UID: \"254ac940-9cf6-4025-aa45-117c82689fa1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.266079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.265914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/254ac940-9cf6-4025-aa45-117c82689fa1-data\") pod \"postgres-69fb4594cf-q68xn\" (UID: \"254ac940-9cf6-4025-aa45-117c82689fa1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.367070 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.367032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxqlw\" (UniqueName: \"kubernetes.io/projected/254ac940-9cf6-4025-aa45-117c82689fa1-kube-api-access-wxqlw\") pod \"postgres-69fb4594cf-q68xn\" (UID: \"254ac940-9cf6-4025-aa45-117c82689fa1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.367306 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.367087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/254ac940-9cf6-4025-aa45-117c82689fa1-data\") pod \"postgres-69fb4594cf-q68xn\" (UID: \"254ac940-9cf6-4025-aa45-117c82689fa1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.367494 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.367477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/254ac940-9cf6-4025-aa45-117c82689fa1-data\") pod \"postgres-69fb4594cf-q68xn\" (UID: \"254ac940-9cf6-4025-aa45-117c82689fa1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.376273 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.376255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxqlw\" (UniqueName: \"kubernetes.io/projected/254ac940-9cf6-4025-aa45-117c82689fa1-kube-api-access-wxqlw\") pod \"postgres-69fb4594cf-q68xn\" (UID: \"254ac940-9cf6-4025-aa45-117c82689fa1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.463776 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.463737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:32.599947 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:32.599914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"]
Jul 15 19:38:32.601337 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:32.601313    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254ac940_9cf6_4025_aa45_117c82689fa1.slice/crio-fde17646429d5b482469b33a05ce5eb8f1603eb01369848a9e532d088b8bc02a WatchSource:0}: Error finding container fde17646429d5b482469b33a05ce5eb8f1603eb01369848a9e532d088b8bc02a: Status 404 returned error can't find the container with id fde17646429d5b482469b33a05ce5eb8f1603eb01369848a9e532d088b8bc02a
Jul 15 19:38:33.421175 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:33.421140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn" event={"ID":"254ac940-9cf6-4025-aa45-117c82689fa1","Type":"ContainerStarted","Data":"fde17646429d5b482469b33a05ce5eb8f1603eb01369848a9e532d088b8bc02a"}
Jul 15 19:38:37.059731 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:37.059693    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:38.453089 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:38.453053    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn" event={"ID":"254ac940-9cf6-4025-aa45-117c82689fa1","Type":"ContainerStarted","Data":"b8d7121bc05351913e94fbb418b75a3435b7eec8b947535f4723317373cadb78"}
Jul 15 19:38:38.453506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:38.453213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:38.471690 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:38.471636    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn" podStartSLOduration=1.018679972 podStartE2EDuration="6.471618485s" totalImagesPullingTime="5.452938513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:38.469339092 +0000 UTC m=+608.547131892" watchObservedRunningTime="2026-07-15 19:38:38.471618485 +0000 UTC m=+608.549411286"
Jul 15 19:38:44.492093 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:44.492062    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-q68xn"
Jul 15 19:38:44.796024 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:44.795940    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:44.965046 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:44.965004    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:45.034447 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.034409    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-f5qb9"]
Jul 15 19:38:45.039506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.039487    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:45.044794 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.044768    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-f5qb9"]
Jul 15 19:38:45.092548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.092518    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9n4r\" (UniqueName: \"kubernetes.io/projected/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4-kube-api-access-r9n4r\") pod \"authorino-5d4b67c446-f5qb9\" (UID: \"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4\") " pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:45.192960 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.192903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9n4r\" (UniqueName: \"kubernetes.io/projected/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4-kube-api-access-r9n4r\") pod \"authorino-5d4b67c446-f5qb9\" (UID: \"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4\") " pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:45.200364 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.200329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9n4r\" (UniqueName: \"kubernetes.io/projected/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4-kube-api-access-r9n4r\") pod \"authorino-5d4b67c446-f5qb9\" (UID: \"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4\") " pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:45.270104 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.270065    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-f5qb9"]
Jul 15 19:38:45.270398 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.270383    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:45.289776 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.289738    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-759745f574-kljtj"]
Jul 15 19:38:45.294428 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.294407    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:45.300747 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.300668    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-759745f574-kljtj"]
Jul 15 19:38:45.394101 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.394069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vbwl\" (UniqueName: \"kubernetes.io/projected/8cef3af5-2d23-4a56-b36f-72d41c0db67b-kube-api-access-2vbwl\") pod \"authorino-759745f574-kljtj\" (UID: \"8cef3af5-2d23-4a56-b36f-72d41c0db67b\") " pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:45.398100 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.398073    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-759745f574-kljtj"]
Jul 15 19:38:45.398377 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:38:45.398356    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2vbwl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-759745f574-kljtj" podUID="8cef3af5-2d23-4a56-b36f-72d41c0db67b"
Jul 15 19:38:45.411121 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.411078    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-f5qb9"]
Jul 15 19:38:45.413583 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:45.413557    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8c136ef_de3c_4c50_88f5_e43c5d52c1e4.slice/crio-04de3915ac19697e57b93103ede55ad0b8e9c2594bd5c69cb4db985dec6112a9 WatchSource:0}: Error finding container 04de3915ac19697e57b93103ede55ad0b8e9c2594bd5c69cb4db985dec6112a9: Status 404 returned error can't find the container with id 04de3915ac19697e57b93103ede55ad0b8e9c2594bd5c69cb4db985dec6112a9
Jul 15 19:38:45.420209 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.420189    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b958659fd-4mg8t"]
Jul 15 19:38:45.431803 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.431775    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.432552 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.432529    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b958659fd-4mg8t"]
Jul 15 19:38:45.434024 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.434005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 19:38:45.495091 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.494870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78pkk\" (UniqueName: \"kubernetes.io/projected/de680400-197a-47ab-b3e5-5b8d041e522a-kube-api-access-78pkk\") pod \"authorino-6b958659fd-4mg8t\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") " pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.495091 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.494921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de680400-197a-47ab-b3e5-5b8d041e522a-tls-cert\") pod \"authorino-6b958659fd-4mg8t\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") " pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.495091 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.495021    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vbwl\" (UniqueName: \"kubernetes.io/projected/8cef3af5-2d23-4a56-b36f-72d41c0db67b-kube-api-access-2vbwl\") pod \"authorino-759745f574-kljtj\" (UID: \"8cef3af5-2d23-4a56-b36f-72d41c0db67b\") " pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:45.495611 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.495372    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-f5qb9" event={"ID":"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4","Type":"ContainerStarted","Data":"04de3915ac19697e57b93103ede55ad0b8e9c2594bd5c69cb4db985dec6112a9"}
Jul 15 19:38:45.495611 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.495415    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:45.503079 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.503049    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vbwl\" (UniqueName: \"kubernetes.io/projected/8cef3af5-2d23-4a56-b36f-72d41c0db67b-kube-api-access-2vbwl\") pod \"authorino-759745f574-kljtj\" (UID: \"8cef3af5-2d23-4a56-b36f-72d41c0db67b\") " pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:45.512398 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.512369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:45.595603 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.595512    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vbwl\" (UniqueName: \"kubernetes.io/projected/8cef3af5-2d23-4a56-b36f-72d41c0db67b-kube-api-access-2vbwl\") pod \"8cef3af5-2d23-4a56-b36f-72d41c0db67b\" (UID: \"8cef3af5-2d23-4a56-b36f-72d41c0db67b\") "
Jul 15 19:38:45.595760 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.595643    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78pkk\" (UniqueName: \"kubernetes.io/projected/de680400-197a-47ab-b3e5-5b8d041e522a-kube-api-access-78pkk\") pod \"authorino-6b958659fd-4mg8t\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") " pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.595760 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.595672    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de680400-197a-47ab-b3e5-5b8d041e522a-tls-cert\") pod \"authorino-6b958659fd-4mg8t\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") " pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.597861 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.597825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cef3af5-2d23-4a56-b36f-72d41c0db67b-kube-api-access-2vbwl" (OuterVolumeSpecName: "kube-api-access-2vbwl") pod "8cef3af5-2d23-4a56-b36f-72d41c0db67b" (UID: "8cef3af5-2d23-4a56-b36f-72d41c0db67b"). InnerVolumeSpecName "kube-api-access-2vbwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:38:45.598167 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.598150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de680400-197a-47ab-b3e5-5b8d041e522a-tls-cert\") pod \"authorino-6b958659fd-4mg8t\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") " pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.602210 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.602188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78pkk\" (UniqueName: \"kubernetes.io/projected/de680400-197a-47ab-b3e5-5b8d041e522a-kube-api-access-78pkk\") pod \"authorino-6b958659fd-4mg8t\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") " pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.696189 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.696158    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vbwl\" (UniqueName: \"kubernetes.io/projected/8cef3af5-2d23-4a56-b36f-72d41c0db67b-kube-api-access-2vbwl\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:38:45.742180 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.742150    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:38:45.880965 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:45.880914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b958659fd-4mg8t"]
Jul 15 19:38:45.882928 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:45.882895    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde680400_197a_47ab_b3e5_5b8d041e522a.slice/crio-de61949d781e838d2ad7e3918f3e28bb1587d0fbbf5d8edcbe8e37a29a8cd1d6 WatchSource:0}: Error finding container de61949d781e838d2ad7e3918f3e28bb1587d0fbbf5d8edcbe8e37a29a8cd1d6: Status 404 returned error can't find the container with id de61949d781e838d2ad7e3918f3e28bb1587d0fbbf5d8edcbe8e37a29a8cd1d6
Jul 15 19:38:46.504749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.504711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-f5qb9" event={"ID":"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4","Type":"ContainerStarted","Data":"6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4"}
Jul 15 19:38:46.505188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.504752    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-f5qb9" podUID="f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" containerName="authorino" containerID="cri-o://6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4" gracePeriod=30
Jul 15 19:38:46.506639 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.506616    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-759745f574-kljtj"
Jul 15 19:38:46.506744 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.506642    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b958659fd-4mg8t" event={"ID":"de680400-197a-47ab-b3e5-5b8d041e522a","Type":"ContainerStarted","Data":"5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa"}
Jul 15 19:38:46.506744 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.506687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b958659fd-4mg8t" event={"ID":"de680400-197a-47ab-b3e5-5b8d041e522a","Type":"ContainerStarted","Data":"de61949d781e838d2ad7e3918f3e28bb1587d0fbbf5d8edcbe8e37a29a8cd1d6"}
Jul 15 19:38:46.518491 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.518444    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-f5qb9" podStartSLOduration=1.16289169 podStartE2EDuration="1.518429602s" totalImagesPullingTime="355.537912ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:46.51776109 +0000 UTC m=+616.595553892" watchObservedRunningTime="2026-07-15 19:38:46.518429602 +0000 UTC m=+616.596222405"
Jul 15 19:38:46.538768 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.538731    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-759745f574-kljtj"]
Jul 15 19:38:46.542114 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.542094    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-759745f574-kljtj"]
Jul 15 19:38:46.553362 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.553306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b958659fd-4mg8t" podStartSLOduration=1.062569116 podStartE2EDuration="1.553285797s" totalImagesPullingTime="490.716681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:46.551915577 +0000 UTC m=+616.629708377" watchObservedRunningTime="2026-07-15 19:38:46.553285797 +0000 UTC m=+616.631078599"
Jul 15 19:38:46.576931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.576883    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8cv6q"]
Jul 15 19:38:46.577150 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.577116    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-8cv6q" podUID="4443b365-5ec6-4cef-904f-3e19c277f8f5" containerName="authorino" containerID="cri-o://d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7" gracePeriod=30
Jul 15 19:38:46.830911 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.830886    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:46.836337 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.836315    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:46.905478 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.905445    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46fkn\" (UniqueName: \"kubernetes.io/projected/4443b365-5ec6-4cef-904f-3e19c277f8f5-kube-api-access-46fkn\") pod \"4443b365-5ec6-4cef-904f-3e19c277f8f5\" (UID: \"4443b365-5ec6-4cef-904f-3e19c277f8f5\") "
Jul 15 19:38:46.907659 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:46.907632    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4443b365-5ec6-4cef-904f-3e19c277f8f5-kube-api-access-46fkn" (OuterVolumeSpecName: "kube-api-access-46fkn") pod "4443b365-5ec6-4cef-904f-3e19c277f8f5" (UID: "4443b365-5ec6-4cef-904f-3e19c277f8f5"). InnerVolumeSpecName "kube-api-access-46fkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:38:47.006611 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.006579    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9n4r\" (UniqueName: \"kubernetes.io/projected/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4-kube-api-access-r9n4r\") pod \"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4\" (UID: \"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4\") "
Jul 15 19:38:47.006811 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.006744    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46fkn\" (UniqueName: \"kubernetes.io/projected/4443b365-5ec6-4cef-904f-3e19c277f8f5-kube-api-access-46fkn\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:38:47.008833 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.008804    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4-kube-api-access-r9n4r" (OuterVolumeSpecName: "kube-api-access-r9n4r") pod "f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" (UID: "f8c136ef-de3c-4c50-88f5-e43c5d52c1e4"). InnerVolumeSpecName "kube-api-access-r9n4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:38:47.107954 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.107863    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9n4r\" (UniqueName: \"kubernetes.io/projected/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4-kube-api-access-r9n4r\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:38:47.513934 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.513904    2564 generic.go:356] "Generic (PLEG): container finished" podID="f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" containerID="6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4" exitCode=0
Jul 15 19:38:47.514442 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.513963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-f5qb9"
Jul 15 19:38:47.514442 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.513996    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-f5qb9" event={"ID":"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4","Type":"ContainerDied","Data":"6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4"}
Jul 15 19:38:47.514442 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.514044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-f5qb9" event={"ID":"f8c136ef-de3c-4c50-88f5-e43c5d52c1e4","Type":"ContainerDied","Data":"04de3915ac19697e57b93103ede55ad0b8e9c2594bd5c69cb4db985dec6112a9"}
Jul 15 19:38:47.514442 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.514065    2564 scope.go:117] "RemoveContainer" containerID="6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4"
Jul 15 19:38:47.515459 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.515247    2564 generic.go:356] "Generic (PLEG): container finished" podID="4443b365-5ec6-4cef-904f-3e19c277f8f5" containerID="d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7" exitCode=0
Jul 15 19:38:47.515459 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.515294    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8cv6q"
Jul 15 19:38:47.515459 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.515335    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8cv6q" event={"ID":"4443b365-5ec6-4cef-904f-3e19c277f8f5","Type":"ContainerDied","Data":"d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7"}
Jul 15 19:38:47.515459 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.515368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8cv6q" event={"ID":"4443b365-5ec6-4cef-904f-3e19c277f8f5","Type":"ContainerDied","Data":"578c14bac25f223227b021258981b7c710d5b2d78630db8bfce390e56bfba9b0"}
Jul 15 19:38:47.523141 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.523125    2564 scope.go:117] "RemoveContainer" containerID="6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4"
Jul 15 19:38:47.523437 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:38:47.523420    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4\": container with ID starting with 6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4 not found: ID does not exist" containerID="6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4"
Jul 15 19:38:47.523491 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.523445    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4"} err="failed to get container status \"6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4\": rpc error: code = NotFound desc = could not find container \"6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4\": container with ID starting with 6da49d750bb902f6d0bef87a7e18c08573629528b448e1952d705f2285b154e4 not found: ID does not exist"
Jul 15 19:38:47.523491 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.523462    2564 scope.go:117] "RemoveContainer" containerID="d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7"
Jul 15 19:38:47.533777 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.533759    2564 scope.go:117] "RemoveContainer" containerID="d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7"
Jul 15 19:38:47.534041 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:38:47.534022    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7\": container with ID starting with d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7 not found: ID does not exist" containerID="d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7"
Jul 15 19:38:47.534096 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.534058    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7"} err="failed to get container status \"d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7\": rpc error: code = NotFound desc = could not find container \"d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7\": container with ID starting with d355bf2e75978df671f11941f268c01ca937fd98985acb0ff3f4dc9dc50b40f7 not found: ID does not exist"
Jul 15 19:38:47.538212 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.538186    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8cv6q"]
Jul 15 19:38:47.541910 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.541887    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8cv6q"]
Jul 15 19:38:47.550578 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.550558    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-f5qb9"]
Jul 15 19:38:47.555383 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:47.555362    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-f5qb9"]
Jul 15 19:38:48.493311 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:48.493276    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4443b365-5ec6-4cef-904f-3e19c277f8f5" path="/var/lib/kubelet/pods/4443b365-5ec6-4cef-904f-3e19c277f8f5/volumes"
Jul 15 19:38:48.493634 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:48.493620    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cef3af5-2d23-4a56-b36f-72d41c0db67b" path="/var/lib/kubelet/pods/8cef3af5-2d23-4a56-b36f-72d41c0db67b/volumes"
Jul 15 19:38:48.493815 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:48.493804    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" path="/var/lib/kubelet/pods/f8c136ef-de3c-4c50-88f5-e43c5d52c1e4/volumes"
Jul 15 19:38:48.763322 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:48.763213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:51.568115 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.568080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:51.673744 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.673658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:51.728763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.728725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:51.769723 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.769688    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-kb567"]
Jul 15 19:38:51.770057 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.770044    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" containerName="authorino"
Jul 15 19:38:51.770100 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.770059    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" containerName="authorino"
Jul 15 19:38:51.770100 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.770086    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4443b365-5ec6-4cef-904f-3e19c277f8f5" containerName="authorino"
Jul 15 19:38:51.770100 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.770091    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4443b365-5ec6-4cef-904f-3e19c277f8f5" containerName="authorino"
Jul 15 19:38:51.770190 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.770149    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4443b365-5ec6-4cef-904f-3e19c277f8f5" containerName="authorino"
Jul 15 19:38:51.770190 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.770156    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c136ef-de3c-4c50-88f5-e43c5d52c1e4" containerName="authorino"
Jul 15 19:38:51.932325 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.932163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-kb567"]
Jul 15 19:38:51.932325 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.932300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:51.943646 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.935349    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tmrbb\""
Jul 15 19:38:51.943646 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.935431    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 19:38:51.943646 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.935559    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 19:38:51.950954 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.950927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6vff\" (UniqueName: \"kubernetes.io/projected/2cb6ec98-9929-4565-b933-cf7a38247372-kube-api-access-h6vff\") pod \"maas-controller-5c87d8b7bd-kb567\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") " pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:51.951093 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.950968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cb6ec98-9929-4565-b933-cf7a38247372-webhook-cert\") pod \"maas-controller-5c87d8b7bd-kb567\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") " pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:51.973270 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:51.973237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:52.052375 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.052335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6vff\" (UniqueName: \"kubernetes.io/projected/2cb6ec98-9929-4565-b933-cf7a38247372-kube-api-access-h6vff\") pod \"maas-controller-5c87d8b7bd-kb567\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") " pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:52.052375 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.052384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cb6ec98-9929-4565-b933-cf7a38247372-webhook-cert\") pod \"maas-controller-5c87d8b7bd-kb567\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") " pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:52.055145 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.055111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cb6ec98-9929-4565-b933-cf7a38247372-webhook-cert\") pod \"maas-controller-5c87d8b7bd-kb567\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") " pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:52.061323 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.061297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6vff\" (UniqueName: \"kubernetes.io/projected/2cb6ec98-9929-4565-b933-cf7a38247372-kube-api-access-h6vff\") pod \"maas-controller-5c87d8b7bd-kb567\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") " pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:52.243460 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.243418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:52.433181 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.433121    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-kb567"]
Jul 15 19:38:52.435506 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:38:52.435052    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb6ec98_9929_4565_b933_cf7a38247372.slice/crio-83460e72f3016a1b2b868552a8bf08c3ddf28634e05edc819bf47d7cb0bf157c WatchSource:0}: Error finding container 83460e72f3016a1b2b868552a8bf08c3ddf28634e05edc819bf47d7cb0bf157c: Status 404 returned error can't find the container with id 83460e72f3016a1b2b868552a8bf08c3ddf28634e05edc819bf47d7cb0bf157c
Jul 15 19:38:52.545266 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:52.545165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c87d8b7bd-kb567" event={"ID":"2cb6ec98-9929-4565-b933-cf7a38247372","Type":"ContainerStarted","Data":"83460e72f3016a1b2b868552a8bf08c3ddf28634e05edc819bf47d7cb0bf157c"}
Jul 15 19:38:54.002161 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:54.002118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:54.103022 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:54.102967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:55.187913 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:55.187871    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:55.290270 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:55.290211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:38:56.571709 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:56.571662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c87d8b7bd-kb567" event={"ID":"2cb6ec98-9929-4565-b933-cf7a38247372","Type":"ContainerStarted","Data":"2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0"}
Jul 15 19:38:56.572124 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:56.571810    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:38:56.588538 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:38:56.588491    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c87d8b7bd-kb567" podStartSLOduration=2.410032553 podStartE2EDuration="5.588475963s" totalImagesPullingTime="3.17844341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:56.586351765 +0000 UTC m=+626.664144567" watchObservedRunningTime="2026-07-15 19:38:56.588475963 +0000 UTC m=+626.666268763"
Jul 15 19:39:07.580620 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:07.580580    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:39:27.613559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:27.613515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:39:28.285277 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.285238    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7874567466-rxfft"]
Jul 15 19:39:28.290524 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.290500    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.292962 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.292941    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 19:39:28.293122 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.293103    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h855w\""
Jul 15 19:39:28.299859 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.299824    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7874567466-rxfft"]
Jul 15 19:39:28.388045 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.388004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-maas-api-tls\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.388045 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.388043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5twg6\" (UniqueName: \"kubernetes.io/projected/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-kube-api-access-5twg6\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.388302 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.388090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-openshift-service-ca\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.488700 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.488664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-maas-api-tls\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.488895 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.488715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5twg6\" (UniqueName: \"kubernetes.io/projected/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-kube-api-access-5twg6\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.488895 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.488773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-openshift-service-ca\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.489683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.489658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-openshift-service-ca\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.491699 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.491672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-maas-api-tls\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.496602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.496580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5twg6\" (UniqueName: \"kubernetes.io/projected/f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa-kube-api-access-5twg6\") pod \"maas-api-7874567466-rxfft\" (UID: \"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa\") " pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.601745 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.601664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:28.749786 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.749760    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7874567466-rxfft"]
Jul 15 19:39:28.751897 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:39:28.751871    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79769cd_b7c1_4ce6_b9b7_f0c05d58ceaa.slice/crio-1bbc5456af0eae145a30007d92cd8f62b1111e485aff677d933ecb833f54d944 WatchSource:0}: Error finding container 1bbc5456af0eae145a30007d92cd8f62b1111e485aff677d933ecb833f54d944: Status 404 returned error can't find the container with id 1bbc5456af0eae145a30007d92cd8f62b1111e485aff677d933ecb833f54d944
Jul 15 19:39:28.753135 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:28.753114    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:39:29.628111 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:29.628049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:39:29.757701 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:29.757663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft" event={"ID":"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa","Type":"ContainerStarted","Data":"1bbc5456af0eae145a30007d92cd8f62b1111e485aff677d933ecb833f54d944"}
Jul 15 19:39:30.764540 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:30.764504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft" event={"ID":"f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa","Type":"ContainerStarted","Data":"7fa3536445bb90d0364f46bcbfdcd4116b5d37f59329143edfe0d5f06d5447e1"}
Jul 15 19:39:30.764939 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:30.764650    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:30.781377 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:30.781331    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft" podStartSLOduration=1.210660541 podStartE2EDuration="2.781313878s" totalImagesPullingTime="1.570653337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:39:30.778853261 +0000 UTC m=+660.856646055" watchObservedRunningTime="2026-07-15 19:39:30.781313878 +0000 UTC m=+660.859106679"
Jul 15 19:39:36.775369 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:36.775339    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7874567466-rxfft"
Jul 15 19:39:37.779523 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:37.779479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:39:37.879815 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:37.879776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:39:56.626133 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.626087    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"]
Jul 15 19:39:56.629914 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.629885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.632250 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.632203    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 19:39:56.632395 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.632350    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-728wb\""
Jul 15 19:39:56.632903 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.632882    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 19:39:56.632980 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.632921    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 19:39:56.641484 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.641457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"]
Jul 15 19:39:56.656288 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.656214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:39:56.746113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.746080    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.746113 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.746115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.746392 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.746163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.746392 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.746183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.746392 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.746263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62cb5290-a72a-4440-bf52-d31d74a9516b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.746392 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.746307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5728s\" (UniqueName: \"kubernetes.io/projected/62cb5290-a72a-4440-bf52-d31d74a9516b-kube-api-access-5728s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.847583 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.847538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5728s\" (UniqueName: \"kubernetes.io/projected/62cb5290-a72a-4440-bf52-d31d74a9516b-kube-api-access-5728s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.847763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.847634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.847763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.847658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.847763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.847733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.847763 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.847757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.848051 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.847783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62cb5290-a72a-4440-bf52-d31d74a9516b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.848111 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.848094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.848167 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.848091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.848251 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.848166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.850166 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.850138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62cb5290-a72a-4440-bf52-d31d74a9516b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.850455 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.850439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62cb5290-a72a-4440-bf52-d31d74a9516b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.861449 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.861425    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5728s\" (UniqueName: \"kubernetes.io/projected/62cb5290-a72a-4440-bf52-d31d74a9516b-kube-api-access-5728s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd\" (UID: \"62cb5290-a72a-4440-bf52-d31d74a9516b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:56.941365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:56.941276    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:39:57.078858 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:57.078818    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"]
Jul 15 19:39:57.081690 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:39:57.081656    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cb5290_a72a_4440_bf52_d31d74a9516b.slice/crio-f7419f1125f01eb14515f3aedfd43dcca1811e2dd73a3b9406e7dbbde785c729 WatchSource:0}: Error finding container f7419f1125f01eb14515f3aedfd43dcca1811e2dd73a3b9406e7dbbde785c729: Status 404 returned error can't find the container with id f7419f1125f01eb14515f3aedfd43dcca1811e2dd73a3b9406e7dbbde785c729
Jul 15 19:39:57.911238 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:57.911169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd" event={"ID":"62cb5290-a72a-4440-bf52-d31d74a9516b","Type":"ContainerStarted","Data":"f7419f1125f01eb14515f3aedfd43dcca1811e2dd73a3b9406e7dbbde785c729"}
Jul 15 19:39:58.917839 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:58.917805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd" event={"ID":"62cb5290-a72a-4440-bf52-d31d74a9516b","Type":"ContainerStarted","Data":"09e86c3e493cbbd7a9f38b0e49b0b03df9baaffd37392a4703d7bab7247eb57d"}
Jul 15 19:39:58.934042 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:39:58.933990    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd" podStartSLOduration=1.815208804 podStartE2EDuration="2.933970025s" totalImagesPullingTime="1.118761221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:39:58.933297169 +0000 UTC m=+689.011089970" watchObservedRunningTime="2026-07-15 19:39:58.933970025 +0000 UTC m=+689.011762830"
Jul 15 19:40:06.942312 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:06.942273    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:40:06.942832 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:06.942331    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:40:06.955251 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:06.955210    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:40:06.971040 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:06.971011    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd"
Jul 15 19:40:18.566797 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:18.566756    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:40:32.662664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:32.662570    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:40:32.762949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:32.762917    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:40:34.307068 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:34.307006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:40:40.766437 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:40:40.766407    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:16.766322 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:41:16.766287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:22.256305 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:41:22.256267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:28.363915 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:41:28.363877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:33.340977 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:41:33.340939    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:43.988994 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:41:43.988953    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:44.044304 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:41:44.044261    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:27.816460 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.816430    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cc87cffbc-kqljb"]
Jul 15 19:42:27.818954 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.818936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:27.826845 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.826821    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cc87cffbc-kqljb"]
Jul 15 19:42:27.835674 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.835642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2367c66-2919-421e-accb-2dda4bc0d13b-tls-cert\") pod \"authorino-6cc87cffbc-kqljb\" (UID: \"b2367c66-2919-421e-accb-2dda4bc0d13b\") " pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:27.835800 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.835681    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88msp\" (UniqueName: \"kubernetes.io/projected/b2367c66-2919-421e-accb-2dda4bc0d13b-kube-api-access-88msp\") pod \"authorino-6cc87cffbc-kqljb\" (UID: \"b2367c66-2919-421e-accb-2dda4bc0d13b\") " pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:27.936327 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.936288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2367c66-2919-421e-accb-2dda4bc0d13b-tls-cert\") pod \"authorino-6cc87cffbc-kqljb\" (UID: \"b2367c66-2919-421e-accb-2dda4bc0d13b\") " pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:27.936517 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.936338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88msp\" (UniqueName: \"kubernetes.io/projected/b2367c66-2919-421e-accb-2dda4bc0d13b-kube-api-access-88msp\") pod \"authorino-6cc87cffbc-kqljb\" (UID: \"b2367c66-2919-421e-accb-2dda4bc0d13b\") " pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:27.938805 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.938778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2367c66-2919-421e-accb-2dda4bc0d13b-tls-cert\") pod \"authorino-6cc87cffbc-kqljb\" (UID: \"b2367c66-2919-421e-accb-2dda4bc0d13b\") " pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:27.943569 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:27.943548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88msp\" (UniqueName: \"kubernetes.io/projected/b2367c66-2919-421e-accb-2dda4bc0d13b-kube-api-access-88msp\") pod \"authorino-6cc87cffbc-kqljb\" (UID: \"b2367c66-2919-421e-accb-2dda4bc0d13b\") " pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:28.129796 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:28.129715    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc87cffbc-kqljb"
Jul 15 19:42:28.138437 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:28.138405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:28.241285 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:28.241162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:28.268404 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:28.268375    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cc87cffbc-kqljb"]
Jul 15 19:42:28.269961 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:42:28.269934    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2367c66_2919_421e_accb_2dda4bc0d13b.slice/crio-2724220352f6e676cb17a852444d335192d3e668a9ffe8f153e9dc3711ea7b8f WatchSource:0}: Error finding container 2724220352f6e676cb17a852444d335192d3e668a9ffe8f153e9dc3711ea7b8f: Status 404 returned error can't find the container with id 2724220352f6e676cb17a852444d335192d3e668a9ffe8f153e9dc3711ea7b8f
Jul 15 19:42:28.715078 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:28.715041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc87cffbc-kqljb" event={"ID":"b2367c66-2919-421e-accb-2dda4bc0d13b","Type":"ContainerStarted","Data":"2724220352f6e676cb17a852444d335192d3e668a9ffe8f153e9dc3711ea7b8f"}
Jul 15 19:42:29.362160 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:29.362120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:29.460534 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:29.460495    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:29.724870 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:29.724832    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc87cffbc-kqljb" event={"ID":"b2367c66-2919-421e-accb-2dda4bc0d13b","Type":"ContainerStarted","Data":"92cbe1ab667722e1b4055c9ff6e914578604642ae9d50c7b3d2bce62ddfbec0f"}
Jul 15 19:42:29.748639 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:29.748575    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cc87cffbc-kqljb" podStartSLOduration=2.242613815 podStartE2EDuration="2.748551038s" totalImagesPullingTime="505.937223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:42:29.7450376 +0000 UTC m=+839.822830482" watchObservedRunningTime="2026-07-15 19:42:29.748551038 +0000 UTC m=+839.826343843"
Jul 15 19:42:29.772593 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:29.772553    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b958659fd-4mg8t"]
Jul 15 19:42:29.772866 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:29.772828    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6b958659fd-4mg8t" podUID="de680400-197a-47ab-b3e5-5b8d041e522a" containerName="authorino" containerID="cri-o://5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa" gracePeriod=30
Jul 15 19:42:30.036981 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.036952    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:42:30.056135 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.056107    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78pkk\" (UniqueName: \"kubernetes.io/projected/de680400-197a-47ab-b3e5-5b8d041e522a-kube-api-access-78pkk\") pod \"de680400-197a-47ab-b3e5-5b8d041e522a\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") "
Jul 15 19:42:30.056351 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.056148    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de680400-197a-47ab-b3e5-5b8d041e522a-tls-cert\") pod \"de680400-197a-47ab-b3e5-5b8d041e522a\" (UID: \"de680400-197a-47ab-b3e5-5b8d041e522a\") "
Jul 15 19:42:30.058618 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.058582    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de680400-197a-47ab-b3e5-5b8d041e522a-kube-api-access-78pkk" (OuterVolumeSpecName: "kube-api-access-78pkk") pod "de680400-197a-47ab-b3e5-5b8d041e522a" (UID: "de680400-197a-47ab-b3e5-5b8d041e522a"). InnerVolumeSpecName "kube-api-access-78pkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:42:30.070619 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.070581    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de680400-197a-47ab-b3e5-5b8d041e522a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "de680400-197a-47ab-b3e5-5b8d041e522a" (UID: "de680400-197a-47ab-b3e5-5b8d041e522a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:42:30.157188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.157152    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78pkk\" (UniqueName: \"kubernetes.io/projected/de680400-197a-47ab-b3e5-5b8d041e522a-kube-api-access-78pkk\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:42:30.157188 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.157184    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de680400-197a-47ab-b3e5-5b8d041e522a-tls-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:42:30.731407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.731366    2564 generic.go:356] "Generic (PLEG): container finished" podID="de680400-197a-47ab-b3e5-5b8d041e522a" containerID="5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa" exitCode=0
Jul 15 19:42:30.731807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.731429    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b958659fd-4mg8t"
Jul 15 19:42:30.731807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.731480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b958659fd-4mg8t" event={"ID":"de680400-197a-47ab-b3e5-5b8d041e522a","Type":"ContainerDied","Data":"5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa"}
Jul 15 19:42:30.731807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.731518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b958659fd-4mg8t" event={"ID":"de680400-197a-47ab-b3e5-5b8d041e522a","Type":"ContainerDied","Data":"de61949d781e838d2ad7e3918f3e28bb1587d0fbbf5d8edcbe8e37a29a8cd1d6"}
Jul 15 19:42:30.731807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.731540    2564 scope.go:117] "RemoveContainer" containerID="5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa"
Jul 15 19:42:30.740802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.740781    2564 scope.go:117] "RemoveContainer" containerID="5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa"
Jul 15 19:42:30.741087 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:42:30.741069    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa\": container with ID starting with 5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa not found: ID does not exist" containerID="5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa"
Jul 15 19:42:30.741150 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.741101    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa"} err="failed to get container status \"5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa\": rpc error: code = NotFound desc = could not find container \"5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa\": container with ID starting with 5d3ccd31e488f94bfe077c25a6ffa71a9bde48b46016be8d5881a3dbc3c17ffa not found: ID does not exist"
Jul 15 19:42:30.746207 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.746181    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b958659fd-4mg8t"]
Jul 15 19:42:30.751121 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:30.751094    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b958659fd-4mg8t"]
Jul 15 19:42:32.494433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:42:32.494392    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de680400-197a-47ab-b3e5-5b8d041e522a" path="/var/lib/kubelet/pods/de680400-197a-47ab-b3e5-5b8d041e522a/volumes"
Jul 15 19:43:30.438380 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:30.438303    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:43:30.442292 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:30.442265    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:43:40.421826 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.421779    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:40.536380 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.536340    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-kb567"]
Jul 15 19:43:40.536652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.536621    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c87d8b7bd-kb567" podUID="2cb6ec98-9929-4565-b933-cf7a38247372" containerName="manager" containerID="cri-o://2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0" gracePeriod=30
Jul 15 19:43:40.793238 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.793190    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:43:40.924438 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.924407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6vff\" (UniqueName: \"kubernetes.io/projected/2cb6ec98-9929-4565-b933-cf7a38247372-kube-api-access-h6vff\") pod \"2cb6ec98-9929-4565-b933-cf7a38247372\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") "
Jul 15 19:43:40.924618 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.924498    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cb6ec98-9929-4565-b933-cf7a38247372-webhook-cert\") pod \"2cb6ec98-9929-4565-b933-cf7a38247372\" (UID: \"2cb6ec98-9929-4565-b933-cf7a38247372\") "
Jul 15 19:43:40.926764 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.926734    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb6ec98-9929-4565-b933-cf7a38247372-kube-api-access-h6vff" (OuterVolumeSpecName: "kube-api-access-h6vff") pod "2cb6ec98-9929-4565-b933-cf7a38247372" (UID: "2cb6ec98-9929-4565-b933-cf7a38247372"). InnerVolumeSpecName "kube-api-access-h6vff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:43:40.926885 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:40.926736    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cb6ec98-9929-4565-b933-cf7a38247372-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2cb6ec98-9929-4565-b933-cf7a38247372" (UID: "2cb6ec98-9929-4565-b933-cf7a38247372"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:43:41.025329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.025245    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6vff\" (UniqueName: \"kubernetes.io/projected/2cb6ec98-9929-4565-b933-cf7a38247372-kube-api-access-h6vff\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:43:41.025329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.025275    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cb6ec98-9929-4565-b933-cf7a38247372-webhook-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:43:41.116102 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.116070    2564 generic.go:356] "Generic (PLEG): container finished" podID="2cb6ec98-9929-4565-b933-cf7a38247372" containerID="2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0" exitCode=0
Jul 15 19:43:41.116283 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.116111    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c87d8b7bd-kb567"
Jul 15 19:43:41.116283 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.116155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c87d8b7bd-kb567" event={"ID":"2cb6ec98-9929-4565-b933-cf7a38247372","Type":"ContainerDied","Data":"2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0"}
Jul 15 19:43:41.116283 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.116211    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c87d8b7bd-kb567" event={"ID":"2cb6ec98-9929-4565-b933-cf7a38247372","Type":"ContainerDied","Data":"83460e72f3016a1b2b868552a8bf08c3ddf28634e05edc819bf47d7cb0bf157c"}
Jul 15 19:43:41.116283 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.116257    2564 scope.go:117] "RemoveContainer" containerID="2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0"
Jul 15 19:43:41.125661 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.125636    2564 scope.go:117] "RemoveContainer" containerID="2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0"
Jul 15 19:43:41.125949 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:43:41.125926    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0\": container with ID starting with 2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0 not found: ID does not exist" containerID="2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0"
Jul 15 19:43:41.126010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.125959    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0"} err="failed to get container status \"2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0\": rpc error: code = NotFound desc = could not find container \"2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0\": container with ID starting with 2f1e4ec26af9f7ef3ad0e27908d50fb027c97089dee2055bc9a330b2a84d61e0 not found: ID does not exist"
Jul 15 19:43:41.136675 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.136642    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-kb567"]
Jul 15 19:43:41.140203 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.140176    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-kb567"]
Jul 15 19:43:41.832041 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.831986    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:41.956948 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.956911    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-vcmtc"]
Jul 15 19:43:41.957336 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.957323    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb6ec98-9929-4565-b933-cf7a38247372" containerName="manager"
Jul 15 19:43:41.957389 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.957338    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb6ec98-9929-4565-b933-cf7a38247372" containerName="manager"
Jul 15 19:43:41.957389 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.957349    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de680400-197a-47ab-b3e5-5b8d041e522a" containerName="authorino"
Jul 15 19:43:41.957389 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.957355    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de680400-197a-47ab-b3e5-5b8d041e522a" containerName="authorino"
Jul 15 19:43:41.957481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.957417    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb6ec98-9929-4565-b933-cf7a38247372" containerName="manager"
Jul 15 19:43:41.957481 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.957427    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de680400-197a-47ab-b3e5-5b8d041e522a" containerName="authorino"
Jul 15 19:43:41.960571 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.960553    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:41.962686 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.962664    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 19:43:41.963350 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.963322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tmrbb\""
Jul 15 19:43:41.963350 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.963336    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 19:43:41.968806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:41.968506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-vcmtc"]
Jul 15 19:43:42.035052 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.035016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mxwm\" (UniqueName: \"kubernetes.io/projected/a68340ee-cf3d-4681-86de-eee569d7d52d-kube-api-access-2mxwm\") pod \"maas-controller-5c87d8b7bd-vcmtc\" (UID: \"a68340ee-cf3d-4681-86de-eee569d7d52d\") " pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.035052 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.035055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a68340ee-cf3d-4681-86de-eee569d7d52d-webhook-cert\") pod \"maas-controller-5c87d8b7bd-vcmtc\" (UID: \"a68340ee-cf3d-4681-86de-eee569d7d52d\") " pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.135652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.135562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mxwm\" (UniqueName: \"kubernetes.io/projected/a68340ee-cf3d-4681-86de-eee569d7d52d-kube-api-access-2mxwm\") pod \"maas-controller-5c87d8b7bd-vcmtc\" (UID: \"a68340ee-cf3d-4681-86de-eee569d7d52d\") " pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.135652 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.135598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a68340ee-cf3d-4681-86de-eee569d7d52d-webhook-cert\") pod \"maas-controller-5c87d8b7bd-vcmtc\" (UID: \"a68340ee-cf3d-4681-86de-eee569d7d52d\") " pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.138365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.138331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a68340ee-cf3d-4681-86de-eee569d7d52d-webhook-cert\") pod \"maas-controller-5c87d8b7bd-vcmtc\" (UID: \"a68340ee-cf3d-4681-86de-eee569d7d52d\") " pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.143416 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.143388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mxwm\" (UniqueName: \"kubernetes.io/projected/a68340ee-cf3d-4681-86de-eee569d7d52d-kube-api-access-2mxwm\") pod \"maas-controller-5c87d8b7bd-vcmtc\" (UID: \"a68340ee-cf3d-4681-86de-eee569d7d52d\") " pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.272987 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.272955    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:42.410028 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.409869    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c87d8b7bd-vcmtc"]
Jul 15 19:43:42.412420 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:43:42.412391    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68340ee_cf3d_4681_86de_eee569d7d52d.slice/crio-8c7d82576a9fa272614a745990d2f43f2fc20dfb2ab47ff0203c7c65b772d07c WatchSource:0}: Error finding container 8c7d82576a9fa272614a745990d2f43f2fc20dfb2ab47ff0203c7c65b772d07c: Status 404 returned error can't find the container with id 8c7d82576a9fa272614a745990d2f43f2fc20dfb2ab47ff0203c7c65b772d07c
Jul 15 19:43:42.493152 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:42.493118    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb6ec98-9929-4565-b933-cf7a38247372" path="/var/lib/kubelet/pods/2cb6ec98-9929-4565-b933-cf7a38247372/volumes"
Jul 15 19:43:43.130329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:43.130205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc" event={"ID":"a68340ee-cf3d-4681-86de-eee569d7d52d","Type":"ContainerStarted","Data":"f0c8f1b70f13ad91581324c5603a30c1f0d98d575eed7f13a9c907f30ecce252"}
Jul 15 19:43:43.130329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:43.130297    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:43:43.130329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:43.130323    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc" event={"ID":"a68340ee-cf3d-4681-86de-eee569d7d52d","Type":"ContainerStarted","Data":"8c7d82576a9fa272614a745990d2f43f2fc20dfb2ab47ff0203c7c65b772d07c"}
Jul 15 19:43:43.146262 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:43.146191    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc" podStartSLOduration=1.7209937800000001 podStartE2EDuration="2.146176454s" totalImagesPullingTime="425.182674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:43:43.143908843 +0000 UTC m=+913.221701643" watchObservedRunningTime="2026-07-15 19:43:43.146176454 +0000 UTC m=+913.223969255"
Jul 15 19:43:54.138514 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:43:54.138479    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c87d8b7bd-vcmtc"
Jul 15 19:44:21.156496 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:21.156429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:21.256543 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:21.256507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:21.400941 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:21.400894    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:27.157508 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:27.157475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:27.209364 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:27.209327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:27.288458 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:27.288409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:27.385459 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:27.385420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:27.518422 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:27.518376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:33.143428 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:33.143387    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:34.381760 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:44:34.381721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:00.141516 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.141431    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"]
Jul 15 19:45:00.149710 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.149676    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"
Jul 15 19:45:00.152178 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.151947    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:45:00.153386 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.153356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"]
Jul 15 19:45:00.161469 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.161448    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"
Jul 15 19:45:00.298480 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.298446    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"]
Jul 15 19:45:00.300557 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:45:00.300530    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82b629c_d912_47aa_8df6_b8df0fb70d54.slice/crio-cb488f7a8751553f0653afc94ddac48a1fcef39c64c59bb372756dc71a381f31 WatchSource:0}: Error finding container cb488f7a8751553f0653afc94ddac48a1fcef39c64c59bb372756dc71a381f31: Status 404 returned error can't find the container with id cb488f7a8751553f0653afc94ddac48a1fcef39c64c59bb372756dc71a381f31
Jul 15 19:45:00.302328 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.302311    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:45:00.567514 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:00.567475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d" event={"ID":"f82b629c-d912-47aa-8df6-b8df0fb70d54","Type":"ContainerStarted","Data":"cb488f7a8751553f0653afc94ddac48a1fcef39c64c59bb372756dc71a381f31"}
Jul 15 19:45:03.586433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:03.586400    2564 generic.go:356] "Generic (PLEG): container finished" podID="f82b629c-d912-47aa-8df6-b8df0fb70d54" containerID="548b51897d4c7be65c63ac0d0563c2a26fa44f1c9f8eb45c0a7076acc22048e0" exitCode=0
Jul 15 19:45:03.586798 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:03.586450    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d" event={"ID":"f82b629c-d912-47aa-8df6-b8df0fb70d54","Type":"ContainerDied","Data":"548b51897d4c7be65c63ac0d0563c2a26fa44f1c9f8eb45c0a7076acc22048e0"}
Jul 15 19:45:04.728398 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:04.728374    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"
Jul 15 19:45:05.598065 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:05.598040    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d"
Jul 15 19:45:05.598275 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:05.598085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735745-8tc6d" event={"ID":"f82b629c-d912-47aa-8df6-b8df0fb70d54","Type":"ContainerDied","Data":"cb488f7a8751553f0653afc94ddac48a1fcef39c64c59bb372756dc71a381f31"}
Jul 15 19:45:05.598275 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:05.598135    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb488f7a8751553f0653afc94ddac48a1fcef39c64c59bb372756dc71a381f31"
Jul 15 19:45:14.168567 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:14.168528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:48.290121 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:48.290074    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:48.341757 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:45:48.341723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:20.485551 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:48:20.485512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:30.478365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:48:30.478337    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:48:30.483666 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:48:30.483643    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:49:20.486414 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:49:20.486368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:20.538848 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:49:20.538812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:04.226678 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:50:04.226635    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:30.528905 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:53:30.528869    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:53:30.533416 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:53:30.533395    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:54:49.789570 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:54:49.789528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:49.914437 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:54:49.914403    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:30.348237 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:30.348147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:34.855740 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:34.855700    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:34.957373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:34.957340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:36.494014 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:36.493979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:41.855923 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:41.855889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:50.130166 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:50.130130    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:55.259046 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:55.259012    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:58.400486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:58.400446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:58.504193 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:58.504161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:58.879348 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:55:58.879313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:04.474444 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:04.474410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:04.575130 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:04.575094    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:04.685749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:04.685715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:04.785951 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:04.785867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:05.777742 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.777694    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:05.865771 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.865739    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"]
Jul 15 19:56:05.866150 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.866137    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f82b629c-d912-47aa-8df6-b8df0fb70d54" containerName="cleanup"
Jul 15 19:56:05.866195 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.866152    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f82b629c-d912-47aa-8df6-b8df0fb70d54" containerName="cleanup"
Jul 15 19:56:05.866247 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.866211    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f82b629c-d912-47aa-8df6-b8df0fb70d54" containerName="cleanup"
Jul 15 19:56:05.868032 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.867997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:05.868364 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.868344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:05.870479 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.870458    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-de95558c-serving-cert\""
Jul 15 19:56:05.881751 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.881730    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"]
Jul 15 19:56:05.936072 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.936038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:05.943303 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.943271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk6lc\" (UniqueName: \"kubernetes.io/projected/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-kube-api-access-mk6lc\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:05.943427 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.943319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-maas-api-tls\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:05.943427 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:05.943352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-openshift-service-ca\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.042049 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.041964    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:06.044100 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.044080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk6lc\" (UniqueName: \"kubernetes.io/projected/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-kube-api-access-mk6lc\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.044160 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.044123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-maas-api-tls\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.044160 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.044145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-openshift-service-ca\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.044761 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.044741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-openshift-service-ca\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.046694 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.046672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-maas-api-tls\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.071032 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.071000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk6lc\" (UniqueName: \"kubernetes.io/projected/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-kube-api-access-mk6lc\") pod \"maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.179823 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.179778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:06.320793 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.320762    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"]
Jul 15 19:56:06.322265 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:56:06.322237    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac1d23a_bc65_4b4b_a2e3_7f6566176f74.slice/crio-aaf223df02b0200c15c959dfdf4a22d12c11cc98c094a006fa63c0c0e9238c45 WatchSource:0}: Error finding container aaf223df02b0200c15c959dfdf4a22d12c11cc98c094a006fa63c0c0e9238c45: Status 404 returned error can't find the container with id aaf223df02b0200c15c959dfdf4a22d12c11cc98c094a006fa63c0c0e9238c45
Jul 15 19:56:06.324011 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.323991    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:56:06.478347 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.478313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:06.579817 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:06.579725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:07.221088 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:07.221051    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp" event={"ID":"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74","Type":"ContainerStarted","Data":"aaf223df02b0200c15c959dfdf4a22d12c11cc98c094a006fa63c0c0e9238c45"}
Jul 15 19:56:08.227882 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:08.227844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp" event={"ID":"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74","Type":"ContainerStarted","Data":"eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b"}
Jul 15 19:56:08.228275 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:08.227980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:08.250007 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:08.246097    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp" podStartSLOduration=1.824298529 podStartE2EDuration="3.24607747s" totalImagesPullingTime="1.421778941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:08.243834901 +0000 UTC m=+1658.321627703" watchObservedRunningTime="2026-07-15 19:56:08.24607747 +0000 UTC m=+1658.323870273"
Jul 15 19:56:08.387042 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:08.387013    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:10.917639 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:10.917605    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"]
Jul 15 19:56:10.920416 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:10.920399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:10.922499 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:10.922477    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:56:10.927264 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:10.927215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"]
Jul 15 19:56:10.992214 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:10.992183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01ce9cd-4305-4c81-977a-f5c3f0500539-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de95558c-hxft9\" (UID: \"e01ce9cd-4305-4c81-977a-f5c3f0500539\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:11.093286 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:11.093246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01ce9cd-4305-4c81-977a-f5c3f0500539-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de95558c-hxft9\" (UID: \"e01ce9cd-4305-4c81-977a-f5c3f0500539\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:11.093951 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:11.093931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01ce9cd-4305-4c81-977a-f5c3f0500539-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de95558c-hxft9\" (UID: \"e01ce9cd-4305-4c81-977a-f5c3f0500539\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:11.233454 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:11.233417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:11.376162 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:11.376134    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"]
Jul 15 19:56:11.377896 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:56:11.377867    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14 WatchSource:0}: Error finding container 184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14: Status 404 returned error can't find the container with id 184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14
Jul 15 19:56:12.250324 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:12.250293    2564 generic.go:356] "Generic (PLEG): container finished" podID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerID="bf5b0d9292dc921f3710dde65fb165f047bba5c65f9543f1f5827d5a4e9a53cf" exitCode=7
Jul 15 19:56:12.250683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:12.250371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" event={"ID":"e01ce9cd-4305-4c81-977a-f5c3f0500539","Type":"ContainerDied","Data":"bf5b0d9292dc921f3710dde65fb165f047bba5c65f9543f1f5827d5a4e9a53cf"}
Jul 15 19:56:12.250683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:12.250419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" event={"ID":"e01ce9cd-4305-4c81-977a-f5c3f0500539","Type":"ContainerStarted","Data":"184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14"}
Jul 15 19:56:12.250683 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:12.250514    2564 scope.go:117] "RemoveContainer" containerID="bf5b0d9292dc921f3710dde65fb165f047bba5c65f9543f1f5827d5a4e9a53cf"
Jul 15 19:56:13.257876 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:13.257838    2564 generic.go:356] "Generic (PLEG): container finished" podID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerID="a5dac22153d8c4d3344c763d63307a1f6ae73dc5f04dd3fd64dfd288da8ce7d7" exitCode=7
Jul 15 19:56:13.258289 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:13.257925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" event={"ID":"e01ce9cd-4305-4c81-977a-f5c3f0500539","Type":"ContainerDied","Data":"a5dac22153d8c4d3344c763d63307a1f6ae73dc5f04dd3fd64dfd288da8ce7d7"}
Jul 15 19:56:13.258289 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:13.257978    2564 scope.go:117] "RemoveContainer" containerID="bf5b0d9292dc921f3710dde65fb165f047bba5c65f9543f1f5827d5a4e9a53cf"
Jul 15 19:56:13.258289 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:13.258088    2564 scope.go:117] "RemoveContainer" containerID="a5dac22153d8c4d3344c763d63307a1f6ae73dc5f04dd3fd64dfd288da8ce7d7"
Jul 15 19:56:13.258436 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:13.258330    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-de95558c-hxft9_odh-ai-gateway-infra(e01ce9cd-4305-4c81-977a-f5c3f0500539)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539"
Jul 15 19:56:14.238854 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:14.238826    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:56:14.265850 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:14.265817    2564 scope.go:117] "RemoveContainer" containerID="a5dac22153d8c4d3344c763d63307a1f6ae73dc5f04dd3fd64dfd288da8ce7d7"
Jul 15 19:56:14.266254 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:14.266042    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-de95558c-hxft9_odh-ai-gateway-infra(e01ce9cd-4305-4c81-977a-f5c3f0500539)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539"
Jul 15 19:56:27.487352 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:27.487319    2564 scope.go:117] "RemoveContainer" containerID="a5dac22153d8c4d3344c763d63307a1f6ae73dc5f04dd3fd64dfd288da8ce7d7"
Jul 15 19:56:28.339853 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:28.339817    2564 generic.go:356] "Generic (PLEG): container finished" podID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerID="0651dba60fda8c42b7c91fe354ee3a9f81297739554b6b6e6e7732f3afaac459" exitCode=0
Jul 15 19:56:28.340018 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:28.339888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" event={"ID":"e01ce9cd-4305-4c81-977a-f5c3f0500539","Type":"ContainerDied","Data":"0651dba60fda8c42b7c91fe354ee3a9f81297739554b6b6e6e7732f3afaac459"}
Jul 15 19:56:28.340018 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:28.339944    2564 scope.go:117] "RemoveContainer" containerID="a5dac22153d8c4d3344c763d63307a1f6ae73dc5f04dd3fd64dfd288da8ce7d7"
Jul 15 19:56:29.364301 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:29.364268    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"]
Jul 15 19:56:29.486322 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:29.486296    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:29.566174 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:29.566144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01ce9cd-4305-4c81-977a-f5c3f0500539-maas-api-service-ca\") pod \"e01ce9cd-4305-4c81-977a-f5c3f0500539\" (UID: \"e01ce9cd-4305-4c81-977a-f5c3f0500539\") "
Jul 15 19:56:29.566533 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:29.566512    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e01ce9cd-4305-4c81-977a-f5c3f0500539-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e01ce9cd-4305-4c81-977a-f5c3f0500539" (UID: "e01ce9cd-4305-4c81-977a-f5c3f0500539"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:56:29.667467 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:29.667355    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01ce9cd-4305-4c81-977a-f5c3f0500539-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:56:30.354547 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:30.354515    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"
Jul 15 19:56:30.354751 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:30.354569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9" event={"ID":"e01ce9cd-4305-4c81-977a-f5c3f0500539","Type":"ContainerDied","Data":"184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14"}
Jul 15 19:56:30.354751 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:30.354628    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14"
Jul 15 19:56:30.381677 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:30.381649    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"]
Jul 15 19:56:30.385148 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:30.385120    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-hxft9"]
Jul 15 19:56:30.493788 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:56:30.493757    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" path="/var/lib/kubelet/pods/e01ce9cd-4305-4c81-977a-f5c3f0500539/volumes"
Jul 15 19:56:35.623108 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:35.623069    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:56:44.196349 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:44.196308    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache]"
Jul 15 19:56:45.632887 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:45.632842    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache]"
Jul 15 19:56:55.682241 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:55.682138    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:56:59.164495 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:56:59.164457    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:00.957676 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.957630    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"]
Jul 15 19:57:00.958535 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.958061    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:00.958535 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.958071    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:00.958535 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.958080    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:00.958535 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.958085    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:00.958535 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.958154    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:00.958535 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.958163    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:00.960279 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.960256    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:00.962307 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.962284    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:57:00.967737 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:00.967713    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"]
Jul 15 19:57:01.063830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:01.063791    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca5214c-aa01-4e2f-a271-406909678ae3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de95558c-qt8t9\" (UID: \"8ca5214c-aa01-4e2f-a271-406909678ae3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:01.165200 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:01.165161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca5214c-aa01-4e2f-a271-406909678ae3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de95558c-qt8t9\" (UID: \"8ca5214c-aa01-4e2f-a271-406909678ae3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:01.165851 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:01.165824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca5214c-aa01-4e2f-a271-406909678ae3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-de95558c-qt8t9\" (UID: \"8ca5214c-aa01-4e2f-a271-406909678ae3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:01.273317 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:01.273209    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:01.410418 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:01.410385    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"]
Jul 15 19:57:01.411803 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:57:01.411761    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca5214c_aa01_4e2f_a271_406909678ae3.slice/crio-8f1997a2810a5e1f85c67685b78f03353275601b969a67c4e4c487e6dfc7ac0f WatchSource:0}: Error finding container 8f1997a2810a5e1f85c67685b78f03353275601b969a67c4e4c487e6dfc7ac0f: Status 404 returned error can't find the container with id 8f1997a2810a5e1f85c67685b78f03353275601b969a67c4e4c487e6dfc7ac0f
Jul 15 19:57:01.516184 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:01.516148    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9" event={"ID":"8ca5214c-aa01-4e2f-a271-406909678ae3","Type":"ContainerStarted","Data":"8f1997a2810a5e1f85c67685b78f03353275601b969a67c4e4c487e6dfc7ac0f"}
Jul 15 19:57:02.523300 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:02.523264    2564 generic.go:356] "Generic (PLEG): container finished" podID="8ca5214c-aa01-4e2f-a271-406909678ae3" containerID="c7679c72daffde712f8ac608f1add5cf77ed667ed8e2656fd0b3016353f43d17" exitCode=0
Jul 15 19:57:02.523675 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:02.523386    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9" event={"ID":"8ca5214c-aa01-4e2f-a271-406909678ae3","Type":"ContainerDied","Data":"c7679c72daffde712f8ac608f1add5cf77ed667ed8e2656fd0b3016353f43d17"}
Jul 15 19:57:03.670207 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:03.670183    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:03.789749 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:03.789660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca5214c-aa01-4e2f-a271-406909678ae3-maas-api-service-ca\") pod \"8ca5214c-aa01-4e2f-a271-406909678ae3\" (UID: \"8ca5214c-aa01-4e2f-a271-406909678ae3\") "
Jul 15 19:57:03.790008 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:03.789984    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ca5214c-aa01-4e2f-a271-406909678ae3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ca5214c-aa01-4e2f-a271-406909678ae3" (UID: "8ca5214c-aa01-4e2f-a271-406909678ae3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:57:03.891014 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:03.890969    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ca5214c-aa01-4e2f-a271-406909678ae3-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:57:04.535846 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:04.535814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9" event={"ID":"8ca5214c-aa01-4e2f-a271-406909678ae3","Type":"ContainerDied","Data":"8f1997a2810a5e1f85c67685b78f03353275601b969a67c4e4c487e6dfc7ac0f"}
Jul 15 19:57:04.535846 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:04.535842    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-de95558c-qt8t9"
Jul 15 19:57:04.535846 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:04.535865    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f1997a2810a5e1f85c67685b78f03353275601b969a67c4e4c487e6dfc7ac0f"
Jul 15 19:57:05.730910 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:05.730867    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:11.528746 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.528704    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"]
Jul 15 19:57:11.529272 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.528971    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp" podUID="5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" containerName="maas-api" containerID="cri-o://eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b" gracePeriod=30
Jul 15 19:57:11.554959 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.554935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:11.773446 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.773418    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:57:11.866088 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.865986    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mk6lc\" (UniqueName: \"kubernetes.io/projected/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-kube-api-access-mk6lc\") pod \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") "
Jul 15 19:57:11.866088 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.866057    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-openshift-service-ca\") pod \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") "
Jul 15 19:57:11.866088 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.866092    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-maas-api-tls\") pod \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\" (UID: \"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74\") "
Jul 15 19:57:11.866455 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.866432    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" (UID: "5ac1d23a-bc65-4b4b-a2e3-7f6566176f74"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:57:11.868556 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.868531    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" (UID: "5ac1d23a-bc65-4b4b-a2e3-7f6566176f74"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:57:11.868636 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.868534    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-kube-api-access-mk6lc" (OuterVolumeSpecName: "kube-api-access-mk6lc") pod "5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" (UID: "5ac1d23a-bc65-4b4b-a2e3-7f6566176f74"). InnerVolumeSpecName "kube-api-access-mk6lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:57:11.967738 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.967688    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk6lc\" (UniqueName: \"kubernetes.io/projected/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-kube-api-access-mk6lc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:57:11.967738 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.967732    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:57:11.967738 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:11.967742    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:57:12.584802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.584769    2564 generic.go:356] "Generic (PLEG): container finished" podID="5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" containerID="eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b" exitCode=0
Jul 15 19:57:12.585278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.584820    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"
Jul 15 19:57:12.585278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.584838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp" event={"ID":"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74","Type":"ContainerDied","Data":"eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b"}
Jul 15 19:57:12.585278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.584898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp" event={"ID":"5ac1d23a-bc65-4b4b-a2e3-7f6566176f74","Type":"ContainerDied","Data":"aaf223df02b0200c15c959dfdf4a22d12c11cc98c094a006fa63c0c0e9238c45"}
Jul 15 19:57:12.585278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.584930    2564 scope.go:117] "RemoveContainer" containerID="eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b"
Jul 15 19:57:12.593998 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.593976    2564 scope.go:117] "RemoveContainer" containerID="eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b"
Jul 15 19:57:12.594309 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:12.594288    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b\": container with ID starting with eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b not found: ID does not exist" containerID="eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b"
Jul 15 19:57:12.594374 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.594320    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b"} err="failed to get container status \"eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b\": rpc error: code = NotFound desc = could not find container \"eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b\": container with ID starting with eb9a105e53e18fea0cb48e424264526bee0d0acb973acd32852ec4cd6eadfd4b not found: ID does not exist"
Jul 15 19:57:12.600545 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.600520    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"]
Jul 15 19:57:12.605144 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:12.605015    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-de95558c-6c4f6b767f-z6cxp"]
Jul 15 19:57:14.204268 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:14.204212    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:14.493109 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:14.493029    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" path="/var/lib/kubelet/pods/5ac1d23a-bc65-4b4b-a2e3-7f6566176f74/volumes"
Jul 15 19:57:15.742820 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:15.742782    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:16.044430 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:16.044344    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:16.207471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:16.207429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:16.308369 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:16.308284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:17.132138 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:17.132088    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:17.132549 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:17.132284    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:22.089472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:22.089429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:22.147413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:22.147383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:22.369807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:22.369725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:22.469602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:22.469570    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:22.569331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:22.569302    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:23.134116 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.134083    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:23.234990 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.234944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:23.856068 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.856029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:23.926272 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926233    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"]
Jul 15 19:57:23.926664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926648    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" containerName="maas-api"
Jul 15 19:57:23.926664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926666    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" containerName="maas-api"
Jul 15 19:57:23.926776 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926682    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:23.926776 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926688    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:23.926776 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926694    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca5214c-aa01-4e2f-a271-406909678ae3" containerName="revoke-keys"
Jul 15 19:57:23.926776 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926700    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca5214c-aa01-4e2f-a271-406909678ae3" containerName="revoke-keys"
Jul 15 19:57:23.926908 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926779    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac1d23a-bc65-4b4b-a2e3-7f6566176f74" containerName="maas-api"
Jul 15 19:57:23.926908 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926790    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca5214c-aa01-4e2f-a271-406909678ae3" containerName="revoke-keys"
Jul 15 19:57:23.926908 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.926803    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e01ce9cd-4305-4c81-977a-f5c3f0500539" containerName="revoke-keys"
Jul 15 19:57:23.930701 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.930680    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:23.932784 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.932760    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2791e51d-serving-cert\""
Jul 15 19:57:23.938669 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.938645    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"]
Jul 15 19:57:23.960350 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:23.960327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:24.077171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.077137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43fb38b7-c634-4105-b274-ce435dd7bc99-maas-api-tls\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.077367 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.077197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43fb38b7-c634-4105-b274-ce435dd7bc99-openshift-service-ca\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.077367 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.077307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lnt9\" (UniqueName: \"kubernetes.io/projected/43fb38b7-c634-4105-b274-ce435dd7bc99-kube-api-access-6lnt9\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.178507 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.178416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnt9\" (UniqueName: \"kubernetes.io/projected/43fb38b7-c634-4105-b274-ce435dd7bc99-kube-api-access-6lnt9\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.178507 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.178488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43fb38b7-c634-4105-b274-ce435dd7bc99-maas-api-tls\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.178958 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.178560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43fb38b7-c634-4105-b274-ce435dd7bc99-openshift-service-ca\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.179288 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.179268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43fb38b7-c634-4105-b274-ce435dd7bc99-openshift-service-ca\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.181161 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.181140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43fb38b7-c634-4105-b274-ce435dd7bc99-maas-api-tls\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.186356 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.186334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnt9\" (UniqueName: \"kubernetes.io/projected/43fb38b7-c634-4105-b274-ce435dd7bc99-kube-api-access-6lnt9\") pod \"maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.244490 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.244453    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:24.383236 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.383180    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"]
Jul 15 19:57:24.384642 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:57:24.384608    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fb38b7_c634_4105_b274_ce435dd7bc99.slice/crio-90f277400dc80c80826965f9cc63201b9311222af5727a6392d5c0b5e59c2f22 WatchSource:0}: Error finding container 90f277400dc80c80826965f9cc63201b9311222af5727a6392d5c0b5e59c2f22: Status 404 returned error can't find the container with id 90f277400dc80c80826965f9cc63201b9311222af5727a6392d5c0b5e59c2f22
Jul 15 19:57:24.489523 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.489493    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:24.590566 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.590533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:24.660365 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:24.660327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l" event={"ID":"43fb38b7-c634-4105-b274-ce435dd7bc99","Type":"ContainerStarted","Data":"90f277400dc80c80826965f9cc63201b9311222af5727a6392d5c0b5e59c2f22"}
Jul 15 19:57:25.667679 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:25.667644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l" event={"ID":"43fb38b7-c634-4105-b274-ce435dd7bc99","Type":"ContainerStarted","Data":"7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2"}
Jul 15 19:57:25.668053 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:25.667709    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:25.689316 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:25.689267    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l" podStartSLOduration=2.302820379 podStartE2EDuration="2.689251783s" totalImagesPullingTime="386.431404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:57:25.688095505 +0000 UTC m=+1735.765888306" watchObservedRunningTime="2026-07-15 19:57:25.689251783 +0000 UTC m=+1735.767044580"
Jul 15 19:57:25.796736 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:25.796700    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:26.520623 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:26.520591    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:28.379016 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.378978    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"]
Jul 15 19:57:28.382902 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.382884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:28.384895 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.384864    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:57:28.388664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.388640    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"]
Jul 15 19:57:28.518830 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.518790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg\" (UID: \"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:28.620256 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.620196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg\" (UID: \"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:28.620788 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.620765    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg\" (UID: \"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:28.696259 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.696168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:28.833791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:28.833763    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"]
Jul 15 19:57:28.835293 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:57:28.835254    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e4a1f3_80c8_4b9e_9d67_2c71b5db521d.slice/crio-23f48585b9c798eee851a174b6441f5e2fab8cfcd6f75066ce723744fc57380b WatchSource:0}: Error finding container 23f48585b9c798eee851a174b6441f5e2fab8cfcd6f75066ce723744fc57380b: Status 404 returned error can't find the container with id 23f48585b9c798eee851a174b6441f5e2fab8cfcd6f75066ce723744fc57380b
Jul 15 19:57:29.162811 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:29.162778    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice/crio-184979ec3136f645145be4628aaa0685cadf53426bdbb7182f50cd88d1b9ad14\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01ce9cd_4305_4c81_977a_f5c3f0500539.slice\": RecentStats: unable to find data in memory cache]"
Jul 15 19:57:29.691408 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:29.691316    2564 generic.go:356] "Generic (PLEG): container finished" podID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerID="0bbd16839090fc279a3408698045a55e134f57e8d19750ed5d75c9404bde1583" exitCode=7
Jul 15 19:57:29.691408 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:29.691365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" event={"ID":"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d","Type":"ContainerDied","Data":"0bbd16839090fc279a3408698045a55e134f57e8d19750ed5d75c9404bde1583"}
Jul 15 19:57:29.691806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:29.691410    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" event={"ID":"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d","Type":"ContainerStarted","Data":"23f48585b9c798eee851a174b6441f5e2fab8cfcd6f75066ce723744fc57380b"}
Jul 15 19:57:29.691806 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:29.691621    2564 scope.go:117] "RemoveContainer" containerID="0bbd16839090fc279a3408698045a55e134f57e8d19750ed5d75c9404bde1583"
Jul 15 19:57:30.698937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:30.698837    2564 generic.go:356] "Generic (PLEG): container finished" podID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerID="faad6c22fed5aea2375e613ec61ca04d9aa1f2277a830bfca5c245078d488426" exitCode=7
Jul 15 19:57:30.698937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:30.698918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" event={"ID":"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d","Type":"ContainerDied","Data":"faad6c22fed5aea2375e613ec61ca04d9aa1f2277a830bfca5c245078d488426"}
Jul 15 19:57:30.699463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:30.698979    2564 scope.go:117] "RemoveContainer" containerID="0bbd16839090fc279a3408698045a55e134f57e8d19750ed5d75c9404bde1583"
Jul 15 19:57:30.699463 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:30.699183    2564 scope.go:117] "RemoveContainer" containerID="faad6c22fed5aea2375e613ec61ca04d9aa1f2277a830bfca5c245078d488426"
Jul 15 19:57:30.699463 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:30.699434    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg_odh-ai-gateway-infra(33e4a1f3-80c8-4b9e-9d67-2c71b5db521d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d"
Jul 15 19:57:31.679991 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:31.679959    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:57:31.706946 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:31.706919    2564 scope.go:117] "RemoveContainer" containerID="faad6c22fed5aea2375e613ec61ca04d9aa1f2277a830bfca5c245078d488426"
Jul 15 19:57:31.707357 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:57:31.707154    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg_odh-ai-gateway-infra(33e4a1f3-80c8-4b9e-9d67-2c71b5db521d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d"
Jul 15 19:57:46.487665 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:46.487631    2564 scope.go:117] "RemoveContainer" containerID="faad6c22fed5aea2375e613ec61ca04d9aa1f2277a830bfca5c245078d488426"
Jul 15 19:57:46.793936 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:46.793898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" event={"ID":"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d","Type":"ContainerStarted","Data":"99aec4a1a9bbb8bfa581f067da7fa1e61e1a15c6d83ed25cde176d7c66a2aaf4"}
Jul 15 19:57:46.808165 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:46.808115    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" podStartSLOduration=18.808098488 podStartE2EDuration="18.808098488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:57:46.807189871 +0000 UTC m=+1756.884982682" watchObservedRunningTime="2026-07-15 19:57:46.808098488 +0000 UTC m=+1756.885891289"
Jul 15 19:57:47.800807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:47.800776    2564 generic.go:356] "Generic (PLEG): container finished" podID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerID="99aec4a1a9bbb8bfa581f067da7fa1e61e1a15c6d83ed25cde176d7c66a2aaf4" exitCode=0
Jul 15 19:57:47.801200 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:47.800848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" event={"ID":"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d","Type":"ContainerDied","Data":"99aec4a1a9bbb8bfa581f067da7fa1e61e1a15c6d83ed25cde176d7c66a2aaf4"}
Jul 15 19:57:47.801200 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:47.800924    2564 scope.go:117] "RemoveContainer" containerID="faad6c22fed5aea2375e613ec61ca04d9aa1f2277a830bfca5c245078d488426"
Jul 15 19:57:47.818165 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:47.818140    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"]
Jul 15 19:57:48.946548 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:48.946520    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:49.003873 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.003836    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d-maas-api-service-ca\") pod \"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d\" (UID: \"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d\") "
Jul 15 19:57:49.004329 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.004305    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" (UID: "33e4a1f3-80c8-4b9e-9d67-2c71b5db521d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:57:49.105334 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.105249    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:57:49.818419 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.818376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg" event={"ID":"33e4a1f3-80c8-4b9e-9d67-2c71b5db521d","Type":"ContainerDied","Data":"23f48585b9c798eee851a174b6441f5e2fab8cfcd6f75066ce723744fc57380b"}
Jul 15 19:57:49.818419 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.818412    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"
Jul 15 19:57:49.818650 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.818440    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23f48585b9c798eee851a174b6441f5e2fab8cfcd6f75066ce723744fc57380b"
Jul 15 19:57:49.848177 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.848125    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"]
Jul 15 19:57:49.850470 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:49.850449    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-kbltg"]
Jul 15 19:57:50.492949 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:57:50.492917    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" path="/var/lib/kubelet/pods/33e4a1f3-80c8-4b9e-9d67-2c71b5db521d/volumes"
Jul 15 19:58:15.206427 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:15.206396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:28.428151 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428068    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"]
Jul 15 19:58:28.428602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428503    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.428602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428516    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.428602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428534    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.428602 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428543    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.428752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428627    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.428752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428634    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.428752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.428640    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:28.431850 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.431834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:28.433840 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.433815    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:58:28.450048 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.449689    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"]
Jul 15 19:58:28.475791 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.475749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8de539-0b55-4235-8c95-1e299e3e5dbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj\" (UID: \"2a8de539-0b55-4235-8c95-1e299e3e5dbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:28.522918 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.522884    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:28.576927 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.576886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8de539-0b55-4235-8c95-1e299e3e5dbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj\" (UID: \"2a8de539-0b55-4235-8c95-1e299e3e5dbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:28.577575 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.577553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8de539-0b55-4235-8c95-1e299e3e5dbc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj\" (UID: \"2a8de539-0b55-4235-8c95-1e299e3e5dbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:28.661992 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.661962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:28.750920 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.750885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:28.892598 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:28.892569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"]
Jul 15 19:58:28.893861 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:58:28.893822    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8de539_0b55_4235_8c95_1e299e3e5dbc.slice/crio-8cf18774a2719c0adedcab2f94361469b9c7478e392e1c46c7d36ce982aac5f9 WatchSource:0}: Error finding container 8cf18774a2719c0adedcab2f94361469b9c7478e392e1c46c7d36ce982aac5f9: Status 404 returned error can't find the container with id 8cf18774a2719c0adedcab2f94361469b9c7478e392e1c46c7d36ce982aac5f9
Jul 15 19:58:29.041643 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:29.041614    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj" event={"ID":"2a8de539-0b55-4235-8c95-1e299e3e5dbc","Type":"ContainerStarted","Data":"8cf18774a2719c0adedcab2f94361469b9c7478e392e1c46c7d36ce982aac5f9"}
Jul 15 19:58:30.048581 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:30.048544    2564 generic.go:356] "Generic (PLEG): container finished" podID="2a8de539-0b55-4235-8c95-1e299e3e5dbc" containerID="5047f7e3a42035cb9afe6c27d79d4045e45dbf38ed4a9b5b6fe64667194da94f" exitCode=0
Jul 15 19:58:30.049033 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:30.048598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj" event={"ID":"2a8de539-0b55-4235-8c95-1e299e3e5dbc","Type":"ContainerDied","Data":"5047f7e3a42035cb9afe6c27d79d4045e45dbf38ed4a9b5b6fe64667194da94f"}
Jul 15 19:58:30.568981 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:30.568944    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:58:30.576022 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:30.575997    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 19:58:31.187848 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:31.187823    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:31.200979 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:31.200953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8de539-0b55-4235-8c95-1e299e3e5dbc-maas-api-service-ca\") pod \"2a8de539-0b55-4235-8c95-1e299e3e5dbc\" (UID: \"2a8de539-0b55-4235-8c95-1e299e3e5dbc\") "
Jul 15 19:58:31.201511 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:31.201460    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a8de539-0b55-4235-8c95-1e299e3e5dbc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a8de539-0b55-4235-8c95-1e299e3e5dbc" (UID: "2a8de539-0b55-4235-8c95-1e299e3e5dbc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:58:31.301916 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:31.301879    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a8de539-0b55-4235-8c95-1e299e3e5dbc-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:58:32.060953 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:32.060921    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj"
Jul 15 19:58:32.061127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:32.060958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj" event={"ID":"2a8de539-0b55-4235-8c95-1e299e3e5dbc","Type":"ContainerDied","Data":"8cf18774a2719c0adedcab2f94361469b9c7478e392e1c46c7d36ce982aac5f9"}
Jul 15 19:58:32.061127 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:32.061032    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cf18774a2719c0adedcab2f94361469b9c7478e392e1c46c7d36ce982aac5f9"
Jul 15 19:58:33.712255 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:33.712196    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"]
Jul 15 19:58:33.712786 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:33.712441    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l" podUID="43fb38b7-c634-4105-b274-ce435dd7bc99" containerName="maas-api" containerID="cri-o://7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2" gracePeriod=30
Jul 15 19:58:33.733172 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:33.733140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:33.971868 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:33.971807    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:58:34.029495 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.029463    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43fb38b7-c634-4105-b274-ce435dd7bc99-openshift-service-ca\") pod \"43fb38b7-c634-4105-b274-ce435dd7bc99\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") "
Jul 15 19:58:34.029677 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.029537    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43fb38b7-c634-4105-b274-ce435dd7bc99-maas-api-tls\") pod \"43fb38b7-c634-4105-b274-ce435dd7bc99\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") "
Jul 15 19:58:34.029677 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.029650    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lnt9\" (UniqueName: \"kubernetes.io/projected/43fb38b7-c634-4105-b274-ce435dd7bc99-kube-api-access-6lnt9\") pod \"43fb38b7-c634-4105-b274-ce435dd7bc99\" (UID: \"43fb38b7-c634-4105-b274-ce435dd7bc99\") "
Jul 15 19:58:34.029838 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.029813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43fb38b7-c634-4105-b274-ce435dd7bc99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43fb38b7-c634-4105-b274-ce435dd7bc99" (UID: "43fb38b7-c634-4105-b274-ce435dd7bc99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:58:34.029918 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.029903    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43fb38b7-c634-4105-b274-ce435dd7bc99-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:58:34.031721 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.031697    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43fb38b7-c634-4105-b274-ce435dd7bc99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43fb38b7-c634-4105-b274-ce435dd7bc99" (UID: "43fb38b7-c634-4105-b274-ce435dd7bc99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:58:34.031828 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.031804    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43fb38b7-c634-4105-b274-ce435dd7bc99-kube-api-access-6lnt9" (OuterVolumeSpecName: "kube-api-access-6lnt9") pod "43fb38b7-c634-4105-b274-ce435dd7bc99" (UID: "43fb38b7-c634-4105-b274-ce435dd7bc99"). InnerVolumeSpecName "kube-api-access-6lnt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:58:34.073331 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.073295    2564 generic.go:356] "Generic (PLEG): container finished" podID="43fb38b7-c634-4105-b274-ce435dd7bc99" containerID="7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2" exitCode=0
Jul 15 19:58:34.073471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.073349    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"
Jul 15 19:58:34.073471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.073388    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l" event={"ID":"43fb38b7-c634-4105-b274-ce435dd7bc99","Type":"ContainerDied","Data":"7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2"}
Jul 15 19:58:34.073471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.073445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l" event={"ID":"43fb38b7-c634-4105-b274-ce435dd7bc99","Type":"ContainerDied","Data":"90f277400dc80c80826965f9cc63201b9311222af5727a6392d5c0b5e59c2f22"}
Jul 15 19:58:34.073471 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.073468    2564 scope.go:117] "RemoveContainer" containerID="7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2"
Jul 15 19:58:34.083503 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.083488    2564 scope.go:117] "RemoveContainer" containerID="7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2"
Jul 15 19:58:34.083738 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:58:34.083720    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2\": container with ID starting with 7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2 not found: ID does not exist" containerID="7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2"
Jul 15 19:58:34.083807 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.083743    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2"} err="failed to get container status \"7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2\": rpc error: code = NotFound desc = could not find container \"7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2\": container with ID starting with 7f896e86ed7af1cc584edb98c06283a0498c5bb7a507086b6c9cd716e8fe6bc2 not found: ID does not exist"
Jul 15 19:58:34.095387 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.095365    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"]
Jul 15 19:58:34.098171 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.098147    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2791e51d-f9b6b6dc4-m2h6l"]
Jul 15 19:58:34.130512 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.130486    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lnt9\" (UniqueName: \"kubernetes.io/projected/43fb38b7-c634-4105-b274-ce435dd7bc99-kube-api-access-6lnt9\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:58:34.130512 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.130511    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43fb38b7-c634-4105-b274-ce435dd7bc99-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:58:34.481546 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.481508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:34.493422 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.493392    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43fb38b7-c634-4105-b274-ce435dd7bc99" path="/var/lib/kubelet/pods/43fb38b7-c634-4105-b274-ce435dd7bc99/volumes"
Jul 15 19:58:34.925379 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:34.925352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:40.049445 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:40.049410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:40.150117 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:40.150082    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:40.290010 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:40.289978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:40.363103 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:40.363010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:41.394780 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.394742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:41.471570 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.471531    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:41.473136 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473108    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"]
Jul 15 19:58:41.473610 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473587    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43fb38b7-c634-4105-b274-ce435dd7bc99" containerName="maas-api"
Jul 15 19:58:41.473610 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473609    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="43fb38b7-c634-4105-b274-ce435dd7bc99" containerName="maas-api"
Jul 15 19:58:41.473789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473620    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:41.473789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473625    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e4a1f3-80c8-4b9e-9d67-2c71b5db521d" containerName="revoke-keys"
Jul 15 19:58:41.473789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473634    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a8de539-0b55-4235-8c95-1e299e3e5dbc" containerName="revoke-keys"
Jul 15 19:58:41.473789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473640    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a8de539-0b55-4235-8c95-1e299e3e5dbc" containerName="revoke-keys"
Jul 15 19:58:41.473789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473721    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a8de539-0b55-4235-8c95-1e299e3e5dbc" containerName="revoke-keys"
Jul 15 19:58:41.473789 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.473731    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="43fb38b7-c634-4105-b274-ce435dd7bc99" containerName="maas-api"
Jul 15 19:58:41.476730 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.476713    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.479061 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.479037    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-37d9d895-serving-cert\""
Jul 15 19:58:41.488189 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.488162    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"]
Jul 15 19:58:41.534486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.534456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:41.599844 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.599813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4be2756e-c454-4211-bfca-f49aaa97beb6-openshift-service-ca\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.600016 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.599893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4be2756e-c454-4211-bfca-f49aaa97beb6-maas-api-tls\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.600016 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.599930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whbcn\" (UniqueName: \"kubernetes.io/projected/4be2756e-c454-4211-bfca-f49aaa97beb6-kube-api-access-whbcn\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.673591 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.673502    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:41.700936 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.700906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4be2756e-c454-4211-bfca-f49aaa97beb6-maas-api-tls\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.701132 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.700955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whbcn\" (UniqueName: \"kubernetes.io/projected/4be2756e-c454-4211-bfca-f49aaa97beb6-kube-api-access-whbcn\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.701132 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.700986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4be2756e-c454-4211-bfca-f49aaa97beb6-openshift-service-ca\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.701641 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.701621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4be2756e-c454-4211-bfca-f49aaa97beb6-openshift-service-ca\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.703497 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.703472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4be2756e-c454-4211-bfca-f49aaa97beb6-maas-api-tls\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.708486 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.708465    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whbcn\" (UniqueName: \"kubernetes.io/projected/4be2756e-c454-4211-bfca-f49aaa97beb6-kube-api-access-whbcn\") pod \"maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.788262 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.788202    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:41.925770 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:41.925734    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"]
Jul 15 19:58:41.927118 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:58:41.927088    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be2756e_c454_4211_bfca_f49aaa97beb6.slice/crio-7f40be87cf5df052fa8f513777f89d0fd7c2d615013ff70faf360f11f31d5c8e WatchSource:0}: Error finding container 7f40be87cf5df052fa8f513777f89d0fd7c2d615013ff70faf360f11f31d5c8e: Status 404 returned error can't find the container with id 7f40be87cf5df052fa8f513777f89d0fd7c2d615013ff70faf360f11f31d5c8e
Jul 15 19:58:42.083534 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:42.083497    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:42.121430 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:42.121399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m" event={"ID":"4be2756e-c454-4211-bfca-f49aaa97beb6","Type":"ContainerStarted","Data":"7f40be87cf5df052fa8f513777f89d0fd7c2d615013ff70faf360f11f31d5c8e"}
Jul 15 19:58:42.185990 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:42.185905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:43.128201 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:43.128149    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m" event={"ID":"4be2756e-c454-4211-bfca-f49aaa97beb6","Type":"ContainerStarted","Data":"7bb3e9d75f37724f57d76572bcd38476ec230a53afba9a30c001f069eabdc2bd"}
Jul 15 19:58:43.128617 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:43.128349    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:43.145850 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:43.145800    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m" podStartSLOduration=1.6195547380000002 podStartE2EDuration="2.14578336s" totalImagesPullingTime="526.228622ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:43.143473679 +0000 UTC m=+1813.221266481" watchObservedRunningTime="2026-07-15 19:58:43.14578336 +0000 UTC m=+1813.223576161"
Jul 15 19:58:43.769168 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:43.769138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:58:46.767259 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.767206    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"]
Jul 15 19:58:46.770934 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.770913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:58:46.774472 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.774447    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:58:46.777413 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.777386    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"]
Jul 15 19:58:46.846254 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.846190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb54ec9-57d6-4049-89e2-8324c97faf5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-37d9d895-l27mr\" (UID: \"9bb54ec9-57d6-4049-89e2-8324c97faf5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:58:46.947559 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.947525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb54ec9-57d6-4049-89e2-8324c97faf5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-37d9d895-l27mr\" (UID: \"9bb54ec9-57d6-4049-89e2-8324c97faf5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:58:46.948158 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:46.948132    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb54ec9-57d6-4049-89e2-8324c97faf5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-37d9d895-l27mr\" (UID: \"9bb54ec9-57d6-4049-89e2-8324c97faf5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:58:47.084307 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:47.084192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:58:47.227382 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:47.227354    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"]
Jul 15 19:58:47.228731 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:58:47.228704    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb54ec9_57d6_4049_89e2_8324c97faf5f.slice/crio-c2296083cd47728f40a85d56e9ed2b2423d6bb31037274fc455d8f4629f926e6 WatchSource:0}: Error finding container c2296083cd47728f40a85d56e9ed2b2423d6bb31037274fc455d8f4629f926e6: Status 404 returned error can't find the container with id c2296083cd47728f40a85d56e9ed2b2423d6bb31037274fc455d8f4629f926e6
Jul 15 19:58:48.161478 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:48.161399    2564 generic.go:356] "Generic (PLEG): container finished" podID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerID="2a3640d373bdb2e5c02851f52f4db28a96e1f19d778f08e6894690f5506b87fc" exitCode=7
Jul 15 19:58:48.161478 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:48.161463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" event={"ID":"9bb54ec9-57d6-4049-89e2-8324c97faf5f","Type":"ContainerDied","Data":"2a3640d373bdb2e5c02851f52f4db28a96e1f19d778f08e6894690f5506b87fc"}
Jul 15 19:58:48.161855 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:48.161506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" event={"ID":"9bb54ec9-57d6-4049-89e2-8324c97faf5f","Type":"ContainerStarted","Data":"c2296083cd47728f40a85d56e9ed2b2423d6bb31037274fc455d8f4629f926e6"}
Jul 15 19:58:48.161855 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:48.161676    2564 scope.go:117] "RemoveContainer" containerID="2a3640d373bdb2e5c02851f52f4db28a96e1f19d778f08e6894690f5506b87fc"
Jul 15 19:58:49.139893 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:49.139819    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:58:49.171502 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:49.171464    2564 generic.go:356] "Generic (PLEG): container finished" podID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerID="5e26c47fa45092d2c53a6eb1d20e66837b8eb79741ef82e8a25a646087e7df31" exitCode=7
Jul 15 19:58:49.172028 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:49.171536    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" event={"ID":"9bb54ec9-57d6-4049-89e2-8324c97faf5f","Type":"ContainerDied","Data":"5e26c47fa45092d2c53a6eb1d20e66837b8eb79741ef82e8a25a646087e7df31"}
Jul 15 19:58:49.172202 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:49.171775    2564 scope.go:117] "RemoveContainer" containerID="5e26c47fa45092d2c53a6eb1d20e66837b8eb79741ef82e8a25a646087e7df31"
Jul 15 19:58:49.172202 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:49.172186    2564 scope.go:117] "RemoveContainer" containerID="2a3640d373bdb2e5c02851f52f4db28a96e1f19d778f08e6894690f5506b87fc"
Jul 15 19:58:49.172341 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:58:49.172322    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-37d9d895-l27mr_odh-ai-gateway-infra(9bb54ec9-57d6-4049-89e2-8324c97faf5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f"
Jul 15 19:58:50.179501 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:58:50.179472    2564 scope.go:117] "RemoveContainer" containerID="5e26c47fa45092d2c53a6eb1d20e66837b8eb79741ef82e8a25a646087e7df31"
Jul 15 19:58:50.179849 ip-10-0-130-66 kubenswrapper[2564]: E0715 19:58:50.179683    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-37d9d895-l27mr_odh-ai-gateway-infra(9bb54ec9-57d6-4049-89e2-8324c97faf5f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f"
Jul 15 19:59:03.487202 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:03.487163    2564 scope.go:117] "RemoveContainer" containerID="5e26c47fa45092d2c53a6eb1d20e66837b8eb79741ef82e8a25a646087e7df31"
Jul 15 19:59:03.569796 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:03.569766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:03.617752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:03.617720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:04.121577 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:04.121493    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:04.223467 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:04.223428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:04.263212 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:04.263178    2564 generic.go:356] "Generic (PLEG): container finished" podID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerID="d89e7ef28346a848c9c5ab67966e401370092fc649252da0ff91edbddd680946" exitCode=0
Jul 15 19:59:04.263407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:04.263257    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" event={"ID":"9bb54ec9-57d6-4049-89e2-8324c97faf5f","Type":"ContainerDied","Data":"d89e7ef28346a848c9c5ab67966e401370092fc649252da0ff91edbddd680946"}
Jul 15 19:59:04.263407 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:04.263330    2564 scope.go:117] "RemoveContainer" containerID="5e26c47fa45092d2c53a6eb1d20e66837b8eb79741ef82e8a25a646087e7df31"
Jul 15 19:59:05.294669 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:05.294637    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"]
Jul 15 19:59:05.408808 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:05.408786    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:59:05.513630 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:05.513598    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb54ec9-57d6-4049-89e2-8324c97faf5f-maas-api-service-ca\") pod \"9bb54ec9-57d6-4049-89e2-8324c97faf5f\" (UID: \"9bb54ec9-57d6-4049-89e2-8324c97faf5f\") "
Jul 15 19:59:05.513918 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:05.513897    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bb54ec9-57d6-4049-89e2-8324c97faf5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bb54ec9-57d6-4049-89e2-8324c97faf5f" (UID: "9bb54ec9-57d6-4049-89e2-8324c97faf5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:59:05.614507 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:05.614417    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb54ec9-57d6-4049-89e2-8324c97faf5f-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:59:06.276621 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:06.276588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr" event={"ID":"9bb54ec9-57d6-4049-89e2-8324c97faf5f","Type":"ContainerDied","Data":"c2296083cd47728f40a85d56e9ed2b2423d6bb31037274fc455d8f4629f926e6"}
Jul 15 19:59:06.276802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:06.276635    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2296083cd47728f40a85d56e9ed2b2423d6bb31037274fc455d8f4629f926e6"
Jul 15 19:59:06.276802 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:06.276601    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"
Jul 15 19:59:06.308906 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:06.308865    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"]
Jul 15 19:59:06.309664 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:06.309644    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-l27mr"]
Jul 15 19:59:06.493506 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:06.493473    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" path="/var/lib/kubelet/pods/9bb54ec9-57d6-4049-89e2-8324c97faf5f/volumes"
Jul 15 19:59:13.280500 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:13.280466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:36.809310 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.809261    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"]
Jul 15 19:59:36.809887 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.809866    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 19:59:36.809931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.809893    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 19:59:36.809931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.809908    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 19:59:36.809931 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.809916    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 19:59:36.810053 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.810041    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 19:59:36.810087 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.810055    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 19:59:36.813595 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.813575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:36.815755 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.815728    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 19:59:36.818813 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.818784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"]
Jul 15 19:59:36.883574 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.883540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44edf938-8e57-4e29-b157-68ef793494d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z\" (UID: \"44edf938-8e57-4e29-b157-68ef793494d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:36.984687 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.984656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44edf938-8e57-4e29-b157-68ef793494d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z\" (UID: \"44edf938-8e57-4e29-b157-68ef793494d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:36.985298 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:36.985280    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44edf938-8e57-4e29-b157-68ef793494d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z\" (UID: \"44edf938-8e57-4e29-b157-68ef793494d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:37.127436 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:37.127349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:37.262752 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:37.262728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"]
Jul 15 19:59:37.264755 ip-10-0-130-66 kubenswrapper[2564]: W0715 19:59:37.264724    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44edf938_8e57_4e29_b157_68ef793494d9.slice/crio-8921d9af689f08e49b6272444f3db669ee825a254639444e43947788b9b88621 WatchSource:0}: Error finding container 8921d9af689f08e49b6272444f3db669ee825a254639444e43947788b9b88621: Status 404 returned error can't find the container with id 8921d9af689f08e49b6272444f3db669ee825a254639444e43947788b9b88621
Jul 15 19:59:37.464643 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:37.464612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z" event={"ID":"44edf938-8e57-4e29-b157-68ef793494d9","Type":"ContainerStarted","Data":"8921d9af689f08e49b6272444f3db669ee825a254639444e43947788b9b88621"}
Jul 15 19:59:38.471562 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:38.471527    2564 generic.go:356] "Generic (PLEG): container finished" podID="44edf938-8e57-4e29-b157-68ef793494d9" containerID="4ebdaba72080f24ec55b9921c241dd1899e9595b22d6b0fe00276f06719a0a2d" exitCode=0
Jul 15 19:59:38.471937 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:38.471616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z" event={"ID":"44edf938-8e57-4e29-b157-68ef793494d9","Type":"ContainerDied","Data":"4ebdaba72080f24ec55b9921c241dd1899e9595b22d6b0fe00276f06719a0a2d"}
Jul 15 19:59:39.623601 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:39.623576    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:39.709232 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:39.709195    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44edf938-8e57-4e29-b157-68ef793494d9-maas-api-service-ca\") pod \"44edf938-8e57-4e29-b157-68ef793494d9\" (UID: \"44edf938-8e57-4e29-b157-68ef793494d9\") "
Jul 15 19:59:39.709607 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:39.709586    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44edf938-8e57-4e29-b157-68ef793494d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44edf938-8e57-4e29-b157-68ef793494d9" (UID: "44edf938-8e57-4e29-b157-68ef793494d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:59:39.810781 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:39.810692    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44edf938-8e57-4e29-b157-68ef793494d9-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:59:40.492309 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:40.492280    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z"
Jul 15 19:59:40.492643 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:40.492620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z" event={"ID":"44edf938-8e57-4e29-b157-68ef793494d9","Type":"ContainerDied","Data":"8921d9af689f08e49b6272444f3db669ee825a254639444e43947788b9b88621"}
Jul 15 19:59:40.492695 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:40.492664    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8921d9af689f08e49b6272444f3db669ee825a254639444e43947788b9b88621"
Jul 15 19:59:47.383951 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.383915    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"]
Jul 15 19:59:47.384436 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.384168    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m" podUID="4be2756e-c454-4211-bfca-f49aaa97beb6" containerName="maas-api" containerID="cri-o://7bb3e9d75f37724f57d76572bcd38476ec230a53afba9a30c001f069eabdc2bd" gracePeriod=30
Jul 15 19:59:47.409755 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.409730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:47.531624 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.531593    2564 generic.go:356] "Generic (PLEG): container finished" podID="4be2756e-c454-4211-bfca-f49aaa97beb6" containerID="7bb3e9d75f37724f57d76572bcd38476ec230a53afba9a30c001f069eabdc2bd" exitCode=0
Jul 15 19:59:47.531751 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.531695    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m" event={"ID":"4be2756e-c454-4211-bfca-f49aaa97beb6","Type":"ContainerDied","Data":"7bb3e9d75f37724f57d76572bcd38476ec230a53afba9a30c001f069eabdc2bd"}
Jul 15 19:59:47.639651 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.639589    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:59:47.789521 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.789485    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4be2756e-c454-4211-bfca-f49aaa97beb6-maas-api-tls\") pod \"4be2756e-c454-4211-bfca-f49aaa97beb6\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") "
Jul 15 19:59:47.789716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.789531    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4be2756e-c454-4211-bfca-f49aaa97beb6-openshift-service-ca\") pod \"4be2756e-c454-4211-bfca-f49aaa97beb6\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") "
Jul 15 19:59:47.789716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.789570    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whbcn\" (UniqueName: \"kubernetes.io/projected/4be2756e-c454-4211-bfca-f49aaa97beb6-kube-api-access-whbcn\") pod \"4be2756e-c454-4211-bfca-f49aaa97beb6\" (UID: \"4be2756e-c454-4211-bfca-f49aaa97beb6\") "
Jul 15 19:59:47.789944 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.789914    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4be2756e-c454-4211-bfca-f49aaa97beb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4be2756e-c454-4211-bfca-f49aaa97beb6" (UID: "4be2756e-c454-4211-bfca-f49aaa97beb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:59:47.791716 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.791693    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4be2756e-c454-4211-bfca-f49aaa97beb6-kube-api-access-whbcn" (OuterVolumeSpecName: "kube-api-access-whbcn") pod "4be2756e-c454-4211-bfca-f49aaa97beb6" (UID: "4be2756e-c454-4211-bfca-f49aaa97beb6"). InnerVolumeSpecName "kube-api-access-whbcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:59:47.791821 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.791725    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4be2756e-c454-4211-bfca-f49aaa97beb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4be2756e-c454-4211-bfca-f49aaa97beb6" (UID: "4be2756e-c454-4211-bfca-f49aaa97beb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:59:47.890670 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.890595    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4be2756e-c454-4211-bfca-f49aaa97beb6-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:59:47.890670 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.890621    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4be2756e-c454-4211-bfca-f49aaa97beb6-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:59:47.890670 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:47.890630    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whbcn\" (UniqueName: \"kubernetes.io/projected/4be2756e-c454-4211-bfca-f49aaa97beb6-kube-api-access-whbcn\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 19:59:48.539433 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:48.539403    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"
Jul 15 19:59:48.539857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:48.539453    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m" event={"ID":"4be2756e-c454-4211-bfca-f49aaa97beb6","Type":"ContainerDied","Data":"7f40be87cf5df052fa8f513777f89d0fd7c2d615013ff70faf360f11f31d5c8e"}
Jul 15 19:59:48.539857 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:48.539510    2564 scope.go:117] "RemoveContainer" containerID="7bb3e9d75f37724f57d76572bcd38476ec230a53afba9a30c001f069eabdc2bd"
Jul 15 19:59:48.556088 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:48.556060    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"]
Jul 15 19:59:48.561643 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:48.561619    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-37d9d895-695bc78c78-vcp9m"]
Jul 15 19:59:50.493561 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:50.493527    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4be2756e-c454-4211-bfca-f49aaa97beb6" path="/var/lib/kubelet/pods/4be2756e-c454-4211-bfca-f49aaa97beb6/volumes"
Jul 15 19:59:51.890017 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:51.889936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:53.236849 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:53.236814    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:53.357373 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:53.357339    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:59.139959 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:59.139923    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:59.208291 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:59.208256    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:59.361097 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:59.361059    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:59.946278 ip-10-0-130-66 kubenswrapper[2564]: I0715 19:59:59.946210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:00.046811 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.046782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:00.128308 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128264    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"]
Jul 15 20:00:00.128690 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128677    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4be2756e-c454-4211-bfca-f49aaa97beb6" containerName="maas-api"
Jul 15 20:00:00.128735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128692    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4be2756e-c454-4211-bfca-f49aaa97beb6" containerName="maas-api"
Jul 15 20:00:00.128735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128705    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44edf938-8e57-4e29-b157-68ef793494d9" containerName="revoke-keys"
Jul 15 20:00:00.128735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128711    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="44edf938-8e57-4e29-b157-68ef793494d9" containerName="revoke-keys"
Jul 15 20:00:00.128735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128732    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 20:00:00.128735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128737    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 20:00:00.128897 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128787    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb54ec9-57d6-4049-89e2-8324c97faf5f" containerName="revoke-keys"
Jul 15 20:00:00.128897 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128796    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4be2756e-c454-4211-bfca-f49aaa97beb6" containerName="maas-api"
Jul 15 20:00:00.128897 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.128804    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="44edf938-8e57-4e29-b157-68ef793494d9" containerName="revoke-keys"
Jul 15 20:00:00.132039 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.132024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"
Jul 15 20:00:00.134169 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.134146    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:00:00.136474 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.136452    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"]
Jul 15 20:00:00.144271 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.144212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"
Jul 15 20:00:00.279246 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.279205    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"]
Jul 15 20:00:00.280649 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:00:00.280625    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd19f1be_7d65_4e89_bbce_7251014e7e37.slice/crio-eccbdb696e98f3b65e7c71a9e112d18e58d012d94cba92e4021fc3b360fcfb5a WatchSource:0}: Error finding container eccbdb696e98f3b65e7c71a9e112d18e58d012d94cba92e4021fc3b360fcfb5a: Status 404 returned error can't find the container with id eccbdb696e98f3b65e7c71a9e112d18e58d012d94cba92e4021fc3b360fcfb5a
Jul 15 20:00:00.475641 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.475608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:00.543441 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.543407    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"]
Jul 15 20:00:00.547583 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.547562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.547843 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.547817    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:00.549513 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.549489    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-033c5f64-serving-cert\""
Jul 15 20:00:00.557194 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.557167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"]
Jul 15 20:00:00.617170 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.617133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:00.617543 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.617519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b" event={"ID":"cd19f1be-7d65-4e89-bbce-7251014e7e37","Type":"ContainerStarted","Data":"25387476b38754bb53e4544f6cfb757769ec13ff773c6e98f113327b6a56f8e5"}
Jul 15 20:00:00.617596 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.617570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b" event={"ID":"cd19f1be-7d65-4e89-bbce-7251014e7e37","Type":"ContainerStarted","Data":"eccbdb696e98f3b65e7c71a9e112d18e58d012d94cba92e4021fc3b360fcfb5a"}
Jul 15 20:00:00.631241 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.631177    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b" podStartSLOduration=0.631163943 podStartE2EDuration="631.163943ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:00.630212575 +0000 UTC m=+1890.708005379" watchObservedRunningTime="2026-07-15 20:00:00.631163943 +0000 UTC m=+1890.708956743"
Jul 15 20:00:00.679040 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.679010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:00.707275 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.707212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4vb2\" (UniqueName: \"kubernetes.io/projected/94eeae17-17bd-4d87-8275-79ace523ce10-kube-api-access-x4vb2\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.707393 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.707322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eeae17-17bd-4d87-8275-79ace523ce10-openshift-service-ca\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.707393 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.707347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eeae17-17bd-4d87-8275-79ace523ce10-maas-api-tls\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.808069 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.808037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4vb2\" (UniqueName: \"kubernetes.io/projected/94eeae17-17bd-4d87-8275-79ace523ce10-kube-api-access-x4vb2\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.808285 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.808081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eeae17-17bd-4d87-8275-79ace523ce10-openshift-service-ca\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.808285 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.808104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eeae17-17bd-4d87-8275-79ace523ce10-maas-api-tls\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.808822 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.808802    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eeae17-17bd-4d87-8275-79ace523ce10-openshift-service-ca\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.810728 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.810711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eeae17-17bd-4d87-8275-79ace523ce10-maas-api-tls\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.817993 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.817972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4vb2\" (UniqueName: \"kubernetes.io/projected/94eeae17-17bd-4d87-8275-79ace523ce10-kube-api-access-x4vb2\") pod \"maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.859958 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.859928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:00.999349 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:00.999320    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"]
Jul 15 20:00:01.000186 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:00:01.000158    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94eeae17_17bd_4d87_8275_79ace523ce10.slice/crio-7a8b620f328fd59b19b1d09b0769025c43e66d27d0c4848b9d938d43b6dc4a6f WatchSource:0}: Error finding container 7a8b620f328fd59b19b1d09b0769025c43e66d27d0c4848b9d938d43b6dc4a6f: Status 404 returned error can't find the container with id 7a8b620f328fd59b19b1d09b0769025c43e66d27d0c4848b9d938d43b6dc4a6f
Jul 15 20:00:01.156478 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:01.156447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:01.256809 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:01.256780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:01.624557 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:01.624475    2564 generic.go:356] "Generic (PLEG): container finished" podID="cd19f1be-7d65-4e89-bbce-7251014e7e37" containerID="25387476b38754bb53e4544f6cfb757769ec13ff773c6e98f113327b6a56f8e5" exitCode=0
Jul 15 20:00:01.624735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:01.624557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b" event={"ID":"cd19f1be-7d65-4e89-bbce-7251014e7e37","Type":"ContainerDied","Data":"25387476b38754bb53e4544f6cfb757769ec13ff773c6e98f113327b6a56f8e5"}
Jul 15 20:00:01.625809 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:01.625787    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52" event={"ID":"94eeae17-17bd-4d87-8275-79ace523ce10","Type":"ContainerStarted","Data":"7a8b620f328fd59b19b1d09b0769025c43e66d27d0c4848b9d938d43b6dc4a6f"}
Jul 15 20:00:02.632797 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:02.632765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52" event={"ID":"94eeae17-17bd-4d87-8275-79ace523ce10","Type":"ContainerStarted","Data":"aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50"}
Jul 15 20:00:02.633242 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:02.632897    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:02.653105 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:02.653058    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52" podStartSLOduration=1.874488398 podStartE2EDuration="2.653043176s" totalImagesPullingTime="778.554778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:02.651395125 +0000 UTC m=+1892.729187939" watchObservedRunningTime="2026-07-15 20:00:02.653043176 +0000 UTC m=+1892.730835976"
Jul 15 20:00:02.758203 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:02.758175    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"
Jul 15 20:00:03.610199 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:03.610167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:03.640159 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:03.640132    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b"
Jul 15 20:00:03.640605 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:03.640162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-46h2b" event={"ID":"cd19f1be-7d65-4e89-bbce-7251014e7e37","Type":"ContainerDied","Data":"eccbdb696e98f3b65e7c71a9e112d18e58d012d94cba92e4021fc3b360fcfb5a"}
Jul 15 20:00:03.640605 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:03.640206    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eccbdb696e98f3b65e7c71a9e112d18e58d012d94cba92e4021fc3b360fcfb5a"
Jul 15 20:00:04.902295 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.902261    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"]
Jul 15 20:00:04.902653 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.902642    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd19f1be-7d65-4e89-bbce-7251014e7e37" containerName="cleanup"
Jul 15 20:00:04.902653 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.902653    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd19f1be-7d65-4e89-bbce-7251014e7e37" containerName="cleanup"
Jul 15 20:00:04.902745 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.902735    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd19f1be-7d65-4e89-bbce-7251014e7e37" containerName="cleanup"
Jul 15 20:00:04.904899 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.904875    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:04.906839 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.906822    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:00:04.911667 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.911643    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"]
Jul 15 20:00:04.942175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:04.942145    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a8130b-44d8-4611-99c1-806aae5761b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj\" (UID: \"85a8130b-44d8-4611-99c1-806aae5761b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:05.043040 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:05.042998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a8130b-44d8-4611-99c1-806aae5761b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj\" (UID: \"85a8130b-44d8-4611-99c1-806aae5761b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:05.043618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:05.043601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a8130b-44d8-4611-99c1-806aae5761b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj\" (UID: \"85a8130b-44d8-4611-99c1-806aae5761b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:05.218666 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:05.218634    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:05.354749 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:05.354719    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"]
Jul 15 20:00:05.356382 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:00:05.356353    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a8130b_44d8_4611_99c1_806aae5761b9.slice/crio-449a2fe5130d7f0a6b89976accb48f5bf042407096a432f2a3fa06242ea76b6e WatchSource:0}: Error finding container 449a2fe5130d7f0a6b89976accb48f5bf042407096a432f2a3fa06242ea76b6e: Status 404 returned error can't find the container with id 449a2fe5130d7f0a6b89976accb48f5bf042407096a432f2a3fa06242ea76b6e
Jul 15 20:00:05.653632 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:05.653605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerStarted","Data":"a6c1adf939cfc9af1bd52a8156dca5c6067c19e31e7e4a0a5a02973616ea4fb2"}
Jul 15 20:00:05.653746 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:05.653652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerStarted","Data":"449a2fe5130d7f0a6b89976accb48f5bf042407096a432f2a3fa06242ea76b6e"}
Jul 15 20:00:06.661357 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:06.661317    2564 generic.go:356] "Generic (PLEG): container finished" podID="85a8130b-44d8-4611-99c1-806aae5761b9" containerID="a6c1adf939cfc9af1bd52a8156dca5c6067c19e31e7e4a0a5a02973616ea4fb2" exitCode=7
Jul 15 20:00:06.661758 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:06.661405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerDied","Data":"a6c1adf939cfc9af1bd52a8156dca5c6067c19e31e7e4a0a5a02973616ea4fb2"}
Jul 15 20:00:06.661758 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:06.661580    2564 scope.go:117] "RemoveContainer" containerID="a6c1adf939cfc9af1bd52a8156dca5c6067c19e31e7e4a0a5a02973616ea4fb2"
Jul 15 20:00:07.669117 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:07.669032    2564 generic.go:356] "Generic (PLEG): container finished" podID="85a8130b-44d8-4611-99c1-806aae5761b9" containerID="4f77501936b4bf24019403c9736b0837246a7258a74ff625057341ce76312405" exitCode=7
Jul 15 20:00:07.669551 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:07.669124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerDied","Data":"4f77501936b4bf24019403c9736b0837246a7258a74ff625057341ce76312405"}
Jul 15 20:00:07.669551 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:07.669181    2564 scope.go:117] "RemoveContainer" containerID="a6c1adf939cfc9af1bd52a8156dca5c6067c19e31e7e4a0a5a02973616ea4fb2"
Jul 15 20:00:07.669551 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:07.669303    2564 scope.go:117] "RemoveContainer" containerID="4f77501936b4bf24019403c9736b0837246a7258a74ff625057341ce76312405"
Jul 15 20:00:07.669551 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:00:07.669527    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj_odh-ai-gateway-infra(85a8130b-44d8-4611-99c1-806aae5761b9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" podUID="85a8130b-44d8-4611-99c1-806aae5761b9"
Jul 15 20:00:08.646066 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:08.646032    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:00:08.680985 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:08.680957    2564 scope.go:117] "RemoveContainer" containerID="4f77501936b4bf24019403c9736b0837246a7258a74ff625057341ce76312405"
Jul 15 20:00:08.681411 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:00:08.681258    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj_odh-ai-gateway-infra(85a8130b-44d8-4611-99c1-806aae5761b9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" podUID="85a8130b-44d8-4611-99c1-806aae5761b9"
Jul 15 20:00:23.486983 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:23.486949    2564 scope.go:117] "RemoveContainer" containerID="4f77501936b4bf24019403c9736b0837246a7258a74ff625057341ce76312405"
Jul 15 20:00:23.773681 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:23.773653    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerStarted","Data":"2ee32bef316f1e679c09d1578fd13cef89fe8f7825c563e461e07c58f95feec5"}
Jul 15 20:00:23.787463 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:23.787421    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" podStartSLOduration=19.787410297 podStartE2EDuration="19.787410297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:05.666247856 +0000 UTC m=+1895.744040656" watchObservedRunningTime="2026-07-15 20:00:23.787410297 +0000 UTC m=+1913.865203099"
Jul 15 20:00:24.781452 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:24.781415    2564 generic.go:356] "Generic (PLEG): container finished" podID="85a8130b-44d8-4611-99c1-806aae5761b9" containerID="2ee32bef316f1e679c09d1578fd13cef89fe8f7825c563e461e07c58f95feec5" exitCode=0
Jul 15 20:00:24.781953 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:24.781491    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerDied","Data":"2ee32bef316f1e679c09d1578fd13cef89fe8f7825c563e461e07c58f95feec5"}
Jul 15 20:00:24.781953 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:24.781554    2564 scope.go:117] "RemoveContainer" containerID="4f77501936b4bf24019403c9736b0837246a7258a74ff625057341ce76312405"
Jul 15 20:00:24.797073 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:24.797043    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"]
Jul 15 20:00:25.921132 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:25.921107    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:26.025887 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.025844    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a8130b-44d8-4611-99c1-806aae5761b9-maas-api-service-ca\") pod \"85a8130b-44d8-4611-99c1-806aae5761b9\" (UID: \"85a8130b-44d8-4611-99c1-806aae5761b9\") "
Jul 15 20:00:26.026261 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.026204    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85a8130b-44d8-4611-99c1-806aae5761b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85a8130b-44d8-4611-99c1-806aae5761b9" (UID: "85a8130b-44d8-4611-99c1-806aae5761b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:26.126896 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.126813    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85a8130b-44d8-4611-99c1-806aae5761b9-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:00:26.795842 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.795746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj" event={"ID":"85a8130b-44d8-4611-99c1-806aae5761b9","Type":"ContainerDied","Data":"449a2fe5130d7f0a6b89976accb48f5bf042407096a432f2a3fa06242ea76b6e"}
Jul 15 20:00:26.795842 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.795776    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"
Jul 15 20:00:26.795842 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.795803    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="449a2fe5130d7f0a6b89976accb48f5bf042407096a432f2a3fa06242ea76b6e"
Jul 15 20:00:26.825039 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.825007    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"]
Jul 15 20:00:26.827324 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:26.827301    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-kqjsj"]
Jul 15 20:00:28.493862 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:00:28.493828    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" path="/var/lib/kubelet/pods/85a8130b-44d8-4611-99c1-806aae5761b9/volumes"
Jul 15 20:01:04.950779 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.950743    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"]
Jul 15 20:01:04.951254 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.951163    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:01:04.951254 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.951175    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:01:04.951254 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.951190    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:01:04.951254 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.951195    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:01:04.951389 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.951274    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:01:04.951389 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.951286    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:01:04.954259 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.954242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:04.956304 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.956280    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:01:04.960474 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.960451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"]
Jul 15 20:01:04.984828 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:04.984796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d431f1-1a55-406c-912e-83c39067fa75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-033c5f64-ldm52\" (UID: \"56d431f1-1a55-406c-912e-83c39067fa75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:05.085909 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:05.085875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d431f1-1a55-406c-912e-83c39067fa75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-033c5f64-ldm52\" (UID: \"56d431f1-1a55-406c-912e-83c39067fa75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:05.086589 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:05.086566    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d431f1-1a55-406c-912e-83c39067fa75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-033c5f64-ldm52\" (UID: \"56d431f1-1a55-406c-912e-83c39067fa75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:05.268490 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:05.268394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:05.406013 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:05.405980    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"]
Jul 15 20:01:05.407289 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:01:05.407262    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d431f1_1a55_406c_912e_83c39067fa75.slice/crio-1e90e05425cfc87949253c929c087b49f187b8ceef3dc6dbb91de6c6b35d59de WatchSource:0}: Error finding container 1e90e05425cfc87949253c929c087b49f187b8ceef3dc6dbb91de6c6b35d59de: Status 404 returned error can't find the container with id 1e90e05425cfc87949253c929c087b49f187b8ceef3dc6dbb91de6c6b35d59de
Jul 15 20:01:06.021550 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:06.021514    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52" event={"ID":"56d431f1-1a55-406c-912e-83c39067fa75","Type":"ContainerStarted","Data":"5d6caf6474316ccc93d099727ff63a3f5e3736f77739474425a6bf4363f89a99"}
Jul 15 20:01:06.021953 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:06.021565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52" event={"ID":"56d431f1-1a55-406c-912e-83c39067fa75","Type":"ContainerStarted","Data":"1e90e05425cfc87949253c929c087b49f187b8ceef3dc6dbb91de6c6b35d59de"}
Jul 15 20:01:06.035515 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:06.035463    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52" podStartSLOduration=2.035446354 podStartE2EDuration="2.035446354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:06.034176018 +0000 UTC m=+1956.111968819" watchObservedRunningTime="2026-07-15 20:01:06.035446354 +0000 UTC m=+1956.113239171"
Jul 15 20:01:07.028841 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:07.028806    2564 generic.go:356] "Generic (PLEG): container finished" podID="56d431f1-1a55-406c-912e-83c39067fa75" containerID="5d6caf6474316ccc93d099727ff63a3f5e3736f77739474425a6bf4363f89a99" exitCode=0
Jul 15 20:01:07.029289 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:07.028903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52" event={"ID":"56d431f1-1a55-406c-912e-83c39067fa75","Type":"ContainerDied","Data":"5d6caf6474316ccc93d099727ff63a3f5e3736f77739474425a6bf4363f89a99"}
Jul 15 20:01:08.175252 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:08.175206    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:08.214925 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:08.214887    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d431f1-1a55-406c-912e-83c39067fa75-maas-api-service-ca\") pod \"56d431f1-1a55-406c-912e-83c39067fa75\" (UID: \"56d431f1-1a55-406c-912e-83c39067fa75\") "
Jul 15 20:01:08.215273 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:08.215251    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d431f1-1a55-406c-912e-83c39067fa75-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56d431f1-1a55-406c-912e-83c39067fa75" (UID: "56d431f1-1a55-406c-912e-83c39067fa75"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:08.316160 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:08.316078    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56d431f1-1a55-406c-912e-83c39067fa75-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:01:09.044837 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:09.044805    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52"
Jul 15 20:01:09.045017 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:09.044844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-033c5f64-ldm52" event={"ID":"56d431f1-1a55-406c-912e-83c39067fa75","Type":"ContainerDied","Data":"1e90e05425cfc87949253c929c087b49f187b8ceef3dc6dbb91de6c6b35d59de"}
Jul 15 20:01:09.045017 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:09.044899    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e90e05425cfc87949253c929c087b49f187b8ceef3dc6dbb91de6c6b35d59de"
Jul 15 20:01:15.511825 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.511787    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"]
Jul 15 20:01:15.512626 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.512591    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52" podUID="94eeae17-17bd-4d87-8275-79ace523ce10" containerName="maas-api" containerID="cri-o://aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50" gracePeriod=30
Jul 15 20:01:15.548816 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.548780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:15.763975 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.763915    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:01:15.887717 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.887687    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4vb2\" (UniqueName: \"kubernetes.io/projected/94eeae17-17bd-4d87-8275-79ace523ce10-kube-api-access-x4vb2\") pod \"94eeae17-17bd-4d87-8275-79ace523ce10\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") "
Jul 15 20:01:15.887929 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.887759    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eeae17-17bd-4d87-8275-79ace523ce10-maas-api-tls\") pod \"94eeae17-17bd-4d87-8275-79ace523ce10\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") "
Jul 15 20:01:15.887929 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.887797    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eeae17-17bd-4d87-8275-79ace523ce10-openshift-service-ca\") pod \"94eeae17-17bd-4d87-8275-79ace523ce10\" (UID: \"94eeae17-17bd-4d87-8275-79ace523ce10\") "
Jul 15 20:01:15.888253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.888193    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94eeae17-17bd-4d87-8275-79ace523ce10-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94eeae17-17bd-4d87-8275-79ace523ce10" (UID: "94eeae17-17bd-4d87-8275-79ace523ce10"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:15.889924 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.889900    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94eeae17-17bd-4d87-8275-79ace523ce10-kube-api-access-x4vb2" (OuterVolumeSpecName: "kube-api-access-x4vb2") pod "94eeae17-17bd-4d87-8275-79ace523ce10" (UID: "94eeae17-17bd-4d87-8275-79ace523ce10"). InnerVolumeSpecName "kube-api-access-x4vb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:01:15.890015 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.889943    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94eeae17-17bd-4d87-8275-79ace523ce10-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94eeae17-17bd-4d87-8275-79ace523ce10" (UID: "94eeae17-17bd-4d87-8275-79ace523ce10"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:15.988711 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.988675    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4vb2\" (UniqueName: \"kubernetes.io/projected/94eeae17-17bd-4d87-8275-79ace523ce10-kube-api-access-x4vb2\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:01:15.988711 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.988706    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eeae17-17bd-4d87-8275-79ace523ce10-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:01:15.988711 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:15.988716    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eeae17-17bd-4d87-8275-79ace523ce10-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:01:16.085492 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.085398    2564 generic.go:356] "Generic (PLEG): container finished" podID="94eeae17-17bd-4d87-8275-79ace523ce10" containerID="aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50" exitCode=0
Jul 15 20:01:16.085492 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.085445    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"
Jul 15 20:01:16.085720 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.085492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52" event={"ID":"94eeae17-17bd-4d87-8275-79ace523ce10","Type":"ContainerDied","Data":"aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50"}
Jul 15 20:01:16.085720 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.085557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52" event={"ID":"94eeae17-17bd-4d87-8275-79ace523ce10","Type":"ContainerDied","Data":"7a8b620f328fd59b19b1d09b0769025c43e66d27d0c4848b9d938d43b6dc4a6f"}
Jul 15 20:01:16.085720 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.085585    2564 scope.go:117] "RemoveContainer" containerID="aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50"
Jul 15 20:01:16.095887 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.095868    2564 scope.go:117] "RemoveContainer" containerID="aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50"
Jul 15 20:01:16.096169 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:01:16.096149    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50\": container with ID starting with aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50 not found: ID does not exist" containerID="aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50"
Jul 15 20:01:16.096211 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.096179    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50"} err="failed to get container status \"aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50\": rpc error: code = NotFound desc = could not find container \"aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50\": container with ID starting with aee14ddeb39a6232d2e47ea8fde05551489878370082a9a75d5ea61eb425fe50 not found: ID does not exist"
Jul 15 20:01:16.107243 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.107199    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"]
Jul 15 20:01:16.110931 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.110907    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-033c5f64-6ff97f47bd-cvg52"]
Jul 15 20:01:16.493018 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:16.492983    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94eeae17-17bd-4d87-8275-79ace523ce10" path="/var/lib/kubelet/pods/94eeae17-17bd-4d87-8275-79ace523ce10/volumes"
Jul 15 20:01:20.407443 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:20.407396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:26.264713 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:26.264653    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:26.327378 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:26.327343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:26.734672 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:26.734639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:26.838827 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:26.838792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:26.934457 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:26.934423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:28.561123 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:28.561090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:34.236458 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:34.236422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:34.298697 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:34.298666    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:34.375157 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:34.375121    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:34.480674 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:34.480639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:34.576756 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:34.576663    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:35.000349 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:01:35.000311    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:01.027442 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:01.027410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:07.134133 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:07.134076    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:07.234679 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:07.234646    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:07.335542 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:07.335505    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:07.435557 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:07.435473    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:24.122484 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:24.122447    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:29.230073 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:29.230034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:29.334807 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:29.334776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:29.431682 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:29.431651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:30.679541 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:30.679512    2564 scope.go:117] "RemoveContainer" containerID="0651dba60fda8c42b7c91fe354ee3a9f81297739554b6b6e6e7732f3afaac459"
Jul 15 20:02:40.021801 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:40.021768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:40.879158 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:40.879122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:47.347618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:47.347584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:47.514954 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:47.514920    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:47.614733 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:47.614647    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:47.715634 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:47.715589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:48.499348 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:48.499319    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:48.596587 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:48.596554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:48.697198 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:48.697172    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:50.409802 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:50.409764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:57.515807 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:57.515719    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:57.617257 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:57.617200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:57.717944 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:02:57.717906    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:03.009621 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:03.009583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:03.109747 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:03.109712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:03.245665 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:03.245624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:03.597409 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:03.597374    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:04.130794 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:04.130746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:06.644078 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644040    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"]
Jul 15 20:03:06.644637 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644618    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56d431f1-1a55-406c-912e-83c39067fa75" containerName="revoke-keys"
Jul 15 20:03:06.644691 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644640    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="56d431f1-1a55-406c-912e-83c39067fa75" containerName="revoke-keys"
Jul 15 20:03:06.644691 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644651    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94eeae17-17bd-4d87-8275-79ace523ce10" containerName="maas-api"
Jul 15 20:03:06.644691 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644661    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94eeae17-17bd-4d87-8275-79ace523ce10" containerName="maas-api"
Jul 15 20:03:06.644788 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644701    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:03:06.644788 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644709    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:03:06.644875 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644812    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85a8130b-44d8-4611-99c1-806aae5761b9" containerName="revoke-keys"
Jul 15 20:03:06.644875 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644826    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94eeae17-17bd-4d87-8275-79ace523ce10" containerName="maas-api"
Jul 15 20:03:06.644875 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.644835    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="56d431f1-1a55-406c-912e-83c39067fa75" containerName="revoke-keys"
Jul 15 20:03:06.651535 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.651506    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:06.652771 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.652747    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.655518 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.655495    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3a3ed256-serving-cert\""
Jul 15 20:03:06.658027 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.658001    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"]
Jul 15 20:03:06.762564 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.762521    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:06.816773 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.816743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2387099-01cb-4bce-b38e-2b9227160f17-maas-api-tls\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.817137 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.817104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4nmq\" (UniqueName: \"kubernetes.io/projected/a2387099-01cb-4bce-b38e-2b9227160f17-kube-api-access-s4nmq\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.817289 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.817149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2387099-01cb-4bce-b38e-2b9227160f17-openshift-service-ca\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.905598 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.905521    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:06.918277 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.918214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4nmq\" (UniqueName: \"kubernetes.io/projected/a2387099-01cb-4bce-b38e-2b9227160f17-kube-api-access-s4nmq\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.918277 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.918284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2387099-01cb-4bce-b38e-2b9227160f17-openshift-service-ca\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.918510 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.918361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2387099-01cb-4bce-b38e-2b9227160f17-maas-api-tls\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.918964 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.918935    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2387099-01cb-4bce-b38e-2b9227160f17-openshift-service-ca\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.920922 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.920901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2387099-01cb-4bce-b38e-2b9227160f17-maas-api-tls\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.940433 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.940402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4nmq\" (UniqueName: \"kubernetes.io/projected/a2387099-01cb-4bce-b38e-2b9227160f17-kube-api-access-s4nmq\") pod \"maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:06.965325 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:06.965281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:07.133158 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.133123    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"]
Jul 15 20:03:07.134056 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:03:07.134028    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2387099_01cb_4bce_b38e_2b9227160f17.slice/crio-0ea90fb11d26092f7f40ce0c71bd692620064a5216954aded9a9bed632313c6b WatchSource:0}: Error finding container 0ea90fb11d26092f7f40ce0c71bd692620064a5216954aded9a9bed632313c6b: Status 404 returned error can't find the container with id 0ea90fb11d26092f7f40ce0c71bd692620064a5216954aded9a9bed632313c6b
Jul 15 20:03:07.135406 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.135385    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:03:07.396875 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.396827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:07.497120 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.497085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:07.729693 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.729649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl" event={"ID":"a2387099-01cb-4bce-b38e-2b9227160f17","Type":"ContainerStarted","Data":"0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da"}
Jul 15 20:03:07.730269 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.729724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl" event={"ID":"a2387099-01cb-4bce-b38e-2b9227160f17","Type":"ContainerStarted","Data":"0ea90fb11d26092f7f40ce0c71bd692620064a5216954aded9a9bed632313c6b"}
Jul 15 20:03:07.730269 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.729973    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:07.748924 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:07.748865    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl" podStartSLOduration=1.310924808 podStartE2EDuration="1.748846961s" totalImagesPullingTime="437.922153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:07.746180418 +0000 UTC m=+2077.823973219" watchObservedRunningTime="2026-07-15 20:03:07.748846961 +0000 UTC m=+2077.826639799"
Jul 15 20:03:11.075788 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:11.075750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:11.804692 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:11.804656    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:11.907297 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:11.907261    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:12.303479 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:12.303448    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:13.743583 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:13.743552    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:18.002586 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.002555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:18.102874 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.102839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:18.393697 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.393603    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"]
Jul 15 20:03:18.398058 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.398034    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:18.400410 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.400390    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:03:18.403541 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.403513    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"]
Jul 15 20:03:18.414768 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.414742    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667caec5-1e01-4089-9064-5901c9236590-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj\" (UID: \"667caec5-1e01-4089-9064-5901c9236590\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:18.516161 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.516121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667caec5-1e01-4089-9064-5901c9236590-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj\" (UID: \"667caec5-1e01-4089-9064-5901c9236590\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:18.517001 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.516972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667caec5-1e01-4089-9064-5901c9236590-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj\" (UID: \"667caec5-1e01-4089-9064-5901c9236590\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:18.710863 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.710838    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:18.849518 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:18.849488    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"]
Jul 15 20:03:18.851341 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:03:18.851317    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667caec5_1e01_4089_9064_5901c9236590.slice/crio-e670bc8ff00c0474ca7a89859620662763ca21434bfc5546b23a7bf781df3b5d WatchSource:0}: Error finding container e670bc8ff00c0474ca7a89859620662763ca21434bfc5546b23a7bf781df3b5d: Status 404 returned error can't find the container with id e670bc8ff00c0474ca7a89859620662763ca21434bfc5546b23a7bf781df3b5d
Jul 15 20:03:19.809075 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:19.809036    2564 generic.go:356] "Generic (PLEG): container finished" podID="667caec5-1e01-4089-9064-5901c9236590" containerID="0517a7e2239e63363413c54eda75355df4c402962da214fdccdab88f83fa3c2a" exitCode=0
Jul 15 20:03:19.809552 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:19.809122    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj" event={"ID":"667caec5-1e01-4089-9064-5901c9236590","Type":"ContainerDied","Data":"0517a7e2239e63363413c54eda75355df4c402962da214fdccdab88f83fa3c2a"}
Jul 15 20:03:19.809552 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:19.809199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj" event={"ID":"667caec5-1e01-4089-9064-5901c9236590","Type":"ContainerStarted","Data":"e670bc8ff00c0474ca7a89859620662763ca21434bfc5546b23a7bf781df3b5d"}
Jul 15 20:03:20.949563 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:20.949539    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:21.033667 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:21.033640    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667caec5-1e01-4089-9064-5901c9236590-maas-api-service-ca\") pod \"667caec5-1e01-4089-9064-5901c9236590\" (UID: \"667caec5-1e01-4089-9064-5901c9236590\") "
Jul 15 20:03:21.033995 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:21.033975    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667caec5-1e01-4089-9064-5901c9236590-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "667caec5-1e01-4089-9064-5901c9236590" (UID: "667caec5-1e01-4089-9064-5901c9236590"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:03:21.135051 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:21.134971    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667caec5-1e01-4089-9064-5901c9236590-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:03:21.820884 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:21.820852    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj"
Jul 15 20:03:21.821098 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:21.820851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj" event={"ID":"667caec5-1e01-4089-9064-5901c9236590","Type":"ContainerDied","Data":"e670bc8ff00c0474ca7a89859620662763ca21434bfc5546b23a7bf781df3b5d"}
Jul 15 20:03:21.821098 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:21.820973    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e670bc8ff00c0474ca7a89859620662763ca21434bfc5546b23a7bf781df3b5d"
Jul 15 20:03:28.810403 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:28.810358    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"]
Jul 15 20:03:28.810875 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:28.810590    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl" podUID="a2387099-01cb-4bce-b38e-2b9227160f17" containerName="maas-api" containerID="cri-o://0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da" gracePeriod=30
Jul 15 20:03:28.838809 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:28.838784    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:29.062096 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.062034    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:29.107909 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.107875    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4nmq\" (UniqueName: \"kubernetes.io/projected/a2387099-01cb-4bce-b38e-2b9227160f17-kube-api-access-s4nmq\") pod \"a2387099-01cb-4bce-b38e-2b9227160f17\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") "
Jul 15 20:03:29.108086 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.107947    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2387099-01cb-4bce-b38e-2b9227160f17-openshift-service-ca\") pod \"a2387099-01cb-4bce-b38e-2b9227160f17\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") "
Jul 15 20:03:29.108086 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.108008    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2387099-01cb-4bce-b38e-2b9227160f17-maas-api-tls\") pod \"a2387099-01cb-4bce-b38e-2b9227160f17\" (UID: \"a2387099-01cb-4bce-b38e-2b9227160f17\") "
Jul 15 20:03:29.108367 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.108335    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2387099-01cb-4bce-b38e-2b9227160f17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2387099-01cb-4bce-b38e-2b9227160f17" (UID: "a2387099-01cb-4bce-b38e-2b9227160f17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:03:29.110192 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.110170    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2387099-01cb-4bce-b38e-2b9227160f17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2387099-01cb-4bce-b38e-2b9227160f17" (UID: "a2387099-01cb-4bce-b38e-2b9227160f17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:03:29.110309 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.110196    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2387099-01cb-4bce-b38e-2b9227160f17-kube-api-access-s4nmq" (OuterVolumeSpecName: "kube-api-access-s4nmq") pod "a2387099-01cb-4bce-b38e-2b9227160f17" (UID: "a2387099-01cb-4bce-b38e-2b9227160f17"). InnerVolumeSpecName "kube-api-access-s4nmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:03:29.209054 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.209015    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4nmq\" (UniqueName: \"kubernetes.io/projected/a2387099-01cb-4bce-b38e-2b9227160f17-kube-api-access-s4nmq\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:03:29.209054 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.209049    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2387099-01cb-4bce-b38e-2b9227160f17-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:03:29.209054 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.209059    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2387099-01cb-4bce-b38e-2b9227160f17-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:03:29.868164 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.868126    2564 generic.go:356] "Generic (PLEG): container finished" podID="a2387099-01cb-4bce-b38e-2b9227160f17" containerID="0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da" exitCode=0
Jul 15 20:03:29.868625 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.868179    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"
Jul 15 20:03:29.868625 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.868234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl" event={"ID":"a2387099-01cb-4bce-b38e-2b9227160f17","Type":"ContainerDied","Data":"0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da"}
Jul 15 20:03:29.868625 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.868290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl" event={"ID":"a2387099-01cb-4bce-b38e-2b9227160f17","Type":"ContainerDied","Data":"0ea90fb11d26092f7f40ce0c71bd692620064a5216954aded9a9bed632313c6b"}
Jul 15 20:03:29.868625 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.868312    2564 scope.go:117] "RemoveContainer" containerID="0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da"
Jul 15 20:03:29.878434 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.878414    2564 scope.go:117] "RemoveContainer" containerID="0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da"
Jul 15 20:03:29.878739 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:03:29.878719    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da\": container with ID starting with 0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da not found: ID does not exist" containerID="0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da"
Jul 15 20:03:29.878834 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.878747    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da"} err="failed to get container status \"0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da\": rpc error: code = NotFound desc = could not find container \"0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da\": container with ID starting with 0fa9f9dd51baa3b1ca59bab15d73235244766cf9df1ddd0bd26b8808b77b17da not found: ID does not exist"
Jul 15 20:03:29.889061 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.889030    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"]
Jul 15 20:03:29.891495 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:29.891473    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3a3ed256-799847f54f-x6mdl"]
Jul 15 20:03:30.494117 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:30.494086    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2387099-01cb-4bce-b38e-2b9227160f17" path="/var/lib/kubelet/pods/a2387099-01cb-4bce-b38e-2b9227160f17/volumes"
Jul 15 20:03:30.619808 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:30.619778    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 20:03:30.624598 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:30.624572    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 20:03:34.187234 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:34.187195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:34.285265 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:34.285215    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:39.581255 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:39.581200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:39.681837 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:39.681792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:39.881646 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:39.881565    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:40.115072 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:40.115037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:40.266152 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:40.266115    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:40.361281 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:40.361243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:40.513787 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:40.513758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:46.316346 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:46.316311    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:46.417531 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:46.417495    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:46.523903 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:46.523860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:46.651893 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:46.651811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:46.749851 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:46.749818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:52.265047 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:52.265014    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:52.365243 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:52.365182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:59.403615 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:59.403584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:59.739639 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:59.739602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:59.972496 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:03:59.972467    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:02.380886 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.380850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:02.481764 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.481715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:02.712211 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.712178    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:02.799290 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.799247    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"]
Jul 15 20:04:02.799897 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.799875    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="667caec5-1e01-4089-9064-5901c9236590" containerName="revoke-keys"
Jul 15 20:04:02.799978 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.799901    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="667caec5-1e01-4089-9064-5901c9236590" containerName="revoke-keys"
Jul 15 20:04:02.799978 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.799927    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2387099-01cb-4bce-b38e-2b9227160f17" containerName="maas-api"
Jul 15 20:04:02.799978 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.799935    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2387099-01cb-4bce-b38e-2b9227160f17" containerName="maas-api"
Jul 15 20:04:02.800073 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.800036    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="667caec5-1e01-4089-9064-5901c9236590" containerName="revoke-keys"
Jul 15 20:04:02.800073 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.800051    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2387099-01cb-4bce-b38e-2b9227160f17" containerName="maas-api"
Jul 15 20:04:02.803657 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.803636    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:02.805978 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.805953    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-12e2475c-serving-cert\""
Jul 15 20:04:02.809781 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.809752    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"]
Jul 15 20:04:02.813944 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.813905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:02.884670 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.884638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:02.914970 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.914928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:02.914970 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.914968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacd2290-1bff-49c6-b0af-cb20cf0cee75-openshift-service-ca\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:02.915260 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:02.914998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcn96\" (UniqueName: \"kubernetes.io/projected/aacd2290-1bff-49c6-b0af-cb20cf0cee75-kube-api-access-pcn96\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.016469 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.016376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.016469 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.016417    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacd2290-1bff-49c6-b0af-cb20cf0cee75-openshift-service-ca\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.016700 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:04:03.016521    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-serving-cert: secret "maas-api-e2e-mt-12e2475c-serving-cert" not found
Jul 15 20:04:03.016700 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.016548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcn96\" (UniqueName: \"kubernetes.io/projected/aacd2290-1bff-49c6-b0af-cb20cf0cee75-kube-api-access-pcn96\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.016700 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:04:03.016591    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls podName:aacd2290-1bff-49c6-b0af-cb20cf0cee75 nodeName:}" failed. No retries permitted until 2026-07-15 20:04:03.516572751 +0000 UTC m=+2133.594365529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls") pod "maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" (UID: "aacd2290-1bff-49c6-b0af-cb20cf0cee75") : secret "maas-api-e2e-mt-12e2475c-serving-cert" not found
Jul 15 20:04:03.016993 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.016975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacd2290-1bff-49c6-b0af-cb20cf0cee75-openshift-service-ca\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.026788 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.026766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcn96\" (UniqueName: \"kubernetes.io/projected/aacd2290-1bff-49c6-b0af-cb20cf0cee75-kube-api-access-pcn96\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.090127 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.090074    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:03.408442 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.408342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:03.509172 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.509134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:03.521404 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.521370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.523871 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.523851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls\") pod \"maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.716181 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.716148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:03.852930 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:03.852894    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"]
Jul 15 20:04:03.853871 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:04:03.853839    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaacd2290_1bff_49c6_b0af_cb20cf0cee75.slice/crio-1189d05631e8050434f6e3b7ed1381e7948a0b3ffaf2ef81cbeed83e9ca3709b WatchSource:0}: Error finding container 1189d05631e8050434f6e3b7ed1381e7948a0b3ffaf2ef81cbeed83e9ca3709b: Status 404 returned error can't find the container with id 1189d05631e8050434f6e3b7ed1381e7948a0b3ffaf2ef81cbeed83e9ca3709b
Jul 15 20:04:04.072587 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:04.072501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" event={"ID":"aacd2290-1bff-49c6-b0af-cb20cf0cee75","Type":"ContainerStarted","Data":"1189d05631e8050434f6e3b7ed1381e7948a0b3ffaf2ef81cbeed83e9ca3709b"}
Jul 15 20:04:04.300796 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:04.300760    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:04.401459 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:04.401372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:04.980257 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:04.980204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:05.079810 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:05.079769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" event={"ID":"aacd2290-1bff-49c6-b0af-cb20cf0cee75","Type":"ContainerStarted","Data":"e8e3ef2e57a212e2319189264d03c7aa74fc834342af1094fba24928389fe333"}
Jul 15 20:04:05.079978 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:05.079833    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:05.081282 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:05.081260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:05.099148 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:05.099103    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" podStartSLOduration=2.835253535 podStartE2EDuration="3.099087204s" totalImagesPullingTime="263.833669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:04:05.097433276 +0000 UTC m=+2135.175226076" watchObservedRunningTime="2026-07-15 20:04:05.099087204 +0000 UTC m=+2135.176880003"
Jul 15 20:04:07.271693 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:07.271647    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:11.097857 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:11.097829    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:11.735680 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:11.735647    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:18.663799 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:18.663762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:30.660114 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.660016    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"]
Jul 15 20:04:30.663571 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.663549    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:30.666132 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.666108    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:04:30.670931 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.670903    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"]
Jul 15 20:04:30.714839 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.714810    2564 scope.go:117] "RemoveContainer" containerID="99aec4a1a9bbb8bfa581f067da7fa1e61e1a15c6d83ed25cde176d7c66a2aaf4"
Jul 15 20:04:30.768098 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.768067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52009507-a6fe-47c4-bc39-ef9207719ef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5\" (UID: \"52009507-a6fe-47c4-bc39-ef9207719ef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:30.869191 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.869161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52009507-a6fe-47c4-bc39-ef9207719ef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5\" (UID: \"52009507-a6fe-47c4-bc39-ef9207719ef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:30.870075 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.870052    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52009507-a6fe-47c4-bc39-ef9207719ef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5\" (UID: \"52009507-a6fe-47c4-bc39-ef9207719ef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:30.977493 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:30.977465    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:31.125557 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:31.125526    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"]
Jul 15 20:04:31.126803 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:04:31.126771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52009507_a6fe_47c4_bc39_ef9207719ef6.slice/crio-d918583f4b7ba1b8d146368f7d336f782eb4fb09480432193d7d9a6036873c64 WatchSource:0}: Error finding container d918583f4b7ba1b8d146368f7d336f782eb4fb09480432193d7d9a6036873c64: Status 404 returned error can't find the container with id d918583f4b7ba1b8d146368f7d336f782eb4fb09480432193d7d9a6036873c64
Jul 15 20:04:31.249475 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:31.249379    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5" event={"ID":"52009507-a6fe-47c4-bc39-ef9207719ef6","Type":"ContainerStarted","Data":"d918583f4b7ba1b8d146368f7d336f782eb4fb09480432193d7d9a6036873c64"}
Jul 15 20:04:32.257156 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:32.257112    2564 generic.go:356] "Generic (PLEG): container finished" podID="52009507-a6fe-47c4-bc39-ef9207719ef6" containerID="8f1e7c78819630d03eba4e419d2728708b6ea3a3061a89f8783ccf33b308ba92" exitCode=0
Jul 15 20:04:32.257555 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:32.257198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5" event={"ID":"52009507-a6fe-47c4-bc39-ef9207719ef6","Type":"ContainerDied","Data":"8f1e7c78819630d03eba4e419d2728708b6ea3a3061a89f8783ccf33b308ba92"}
Jul 15 20:04:33.401830 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:33.401804    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:33.493285 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:33.493254    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52009507-a6fe-47c4-bc39-ef9207719ef6-maas-api-service-ca\") pod \"52009507-a6fe-47c4-bc39-ef9207719ef6\" (UID: \"52009507-a6fe-47c4-bc39-ef9207719ef6\") "
Jul 15 20:04:33.493610 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:33.493589    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52009507-a6fe-47c4-bc39-ef9207719ef6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52009507-a6fe-47c4-bc39-ef9207719ef6" (UID: "52009507-a6fe-47c4-bc39-ef9207719ef6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:04:33.594191 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:33.594076    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52009507-a6fe-47c4-bc39-ef9207719ef6-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:04:34.270621 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:34.270585    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5"
Jul 15 20:04:34.270816 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:34.270625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5" event={"ID":"52009507-a6fe-47c4-bc39-ef9207719ef6","Type":"ContainerDied","Data":"d918583f4b7ba1b8d146368f7d336f782eb4fb09480432193d7d9a6036873c64"}
Jul 15 20:04:34.270816 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:34.270704    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d918583f4b7ba1b8d146368f7d336f782eb4fb09480432193d7d9a6036873c64"
Jul 15 20:04:46.126352 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.126316    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"]
Jul 15 20:04:46.126777 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.126570    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" podUID="aacd2290-1bff-49c6-b0af-cb20cf0cee75" containerName="maas-api" containerID="cri-o://e8e3ef2e57a212e2319189264d03c7aa74fc834342af1094fba24928389fe333" gracePeriod=30
Jul 15 20:04:46.145698 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.145666    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:46.350678 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.350642    2564 generic.go:356] "Generic (PLEG): container finished" podID="aacd2290-1bff-49c6-b0af-cb20cf0cee75" containerID="e8e3ef2e57a212e2319189264d03c7aa74fc834342af1094fba24928389fe333" exitCode=0
Jul 15 20:04:46.350848 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.350720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" event={"ID":"aacd2290-1bff-49c6-b0af-cb20cf0cee75","Type":"ContainerDied","Data":"e8e3ef2e57a212e2319189264d03c7aa74fc834342af1094fba24928389fe333"}
Jul 15 20:04:46.379892 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.379836    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:46.520802 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.520773    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls\") pod \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") "
Jul 15 20:04:46.520999 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.520817    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacd2290-1bff-49c6-b0af-cb20cf0cee75-openshift-service-ca\") pod \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") "
Jul 15 20:04:46.520999 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.520931    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcn96\" (UniqueName: \"kubernetes.io/projected/aacd2290-1bff-49c6-b0af-cb20cf0cee75-kube-api-access-pcn96\") pod \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\" (UID: \"aacd2290-1bff-49c6-b0af-cb20cf0cee75\") "
Jul 15 20:04:46.521129 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.521115    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aacd2290-1bff-49c6-b0af-cb20cf0cee75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aacd2290-1bff-49c6-b0af-cb20cf0cee75" (UID: "aacd2290-1bff-49c6-b0af-cb20cf0cee75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:04:46.521380 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.521360    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacd2290-1bff-49c6-b0af-cb20cf0cee75-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:04:46.523060 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.523036    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aacd2290-1bff-49c6-b0af-cb20cf0cee75" (UID: "aacd2290-1bff-49c6-b0af-cb20cf0cee75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:04:46.523126 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.523104    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aacd2290-1bff-49c6-b0af-cb20cf0cee75-kube-api-access-pcn96" (OuterVolumeSpecName: "kube-api-access-pcn96") pod "aacd2290-1bff-49c6-b0af-cb20cf0cee75" (UID: "aacd2290-1bff-49c6-b0af-cb20cf0cee75"). InnerVolumeSpecName "kube-api-access-pcn96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:04:46.622512 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.622477    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacd2290-1bff-49c6-b0af-cb20cf0cee75-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:04:46.622512 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:46.622506    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcn96\" (UniqueName: \"kubernetes.io/projected/aacd2290-1bff-49c6-b0af-cb20cf0cee75-kube-api-access-pcn96\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:04:47.358160 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:47.358132    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"
Jul 15 20:04:47.358566 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:47.358129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk" event={"ID":"aacd2290-1bff-49c6-b0af-cb20cf0cee75","Type":"ContainerDied","Data":"1189d05631e8050434f6e3b7ed1381e7948a0b3ffaf2ef81cbeed83e9ca3709b"}
Jul 15 20:04:47.358566 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:47.358292    2564 scope.go:117] "RemoveContainer" containerID="e8e3ef2e57a212e2319189264d03c7aa74fc834342af1094fba24928389fe333"
Jul 15 20:04:47.378917 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:47.378881    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"]
Jul 15 20:04:47.385398 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:47.385378    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12e2475c-5c4d9b947c-2prnk"]
Jul 15 20:04:48.493084 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:48.493053    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aacd2290-1bff-49c6-b0af-cb20cf0cee75" path="/var/lib/kubelet/pods/aacd2290-1bff-49c6-b0af-cb20cf0cee75/volumes"
Jul 15 20:04:50.754039 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:50.753994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:53.074351 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:53.074310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:04:58.138995 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:04:58.138948    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:30.737327 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:30.737296    2564 scope.go:117] "RemoveContainer" containerID="d89e7ef28346a848c9c5ab67966e401370092fc649252da0ff91edbddd680946"
Jul 15 20:05:34.937902 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:34.937818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:44.555688 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:44.555627    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:45.019910 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:45.019870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:50.534620 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:50.534580    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:50.599968 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:50.599931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:50.818880 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:50.818765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:51.074946 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:51.074858    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:52.290455 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:52.290370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:52.382587 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:52.382553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:52.482600 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:52.482570    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:53.312968 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:53.312932    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:53.425263 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:53.425171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:53.547081 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:53.547043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:05:56.357085 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:05:56.357052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:02.038819 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:02.038778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:02.140362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:02.140327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:07.055518 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:07.055481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:07.162844 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:07.162812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:07.256302 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:07.256239    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:17.812867 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:17.812834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:18.160748 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:18.160652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:30.763329 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:30.763296    2564 scope.go:117] "RemoveContainer" containerID="2ee32bef316f1e679c09d1578fd13cef89fe8f7825c563e461e07c58f95feec5"
Jul 15 20:06:39.198487 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:39.198431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:45.414658 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:45.414614    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:45.515625 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:45.515588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:45.913106 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:45.913071    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:46.314108 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:46.314071    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:46.421960 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:46.421929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:50.286886 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.286849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:50.332117 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332083    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"]
Jul 15 20:06:50.332538 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332523    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52009507-a6fe-47c4-bc39-ef9207719ef6" containerName="revoke-keys"
Jul 15 20:06:50.332594 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332540    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="52009507-a6fe-47c4-bc39-ef9207719ef6" containerName="revoke-keys"
Jul 15 20:06:50.332594 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332557    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aacd2290-1bff-49c6-b0af-cb20cf0cee75" containerName="maas-api"
Jul 15 20:06:50.332594 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332563    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aacd2290-1bff-49c6-b0af-cb20cf0cee75" containerName="maas-api"
Jul 15 20:06:50.332701 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332635    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aacd2290-1bff-49c6-b0af-cb20cf0cee75" containerName="maas-api"
Jul 15 20:06:50.332701 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.332643    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="52009507-a6fe-47c4-bc39-ef9207719ef6" containerName="revoke-keys"
Jul 15 20:06:50.335824 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.335801    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.337781 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.337758    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a86716-serving-cert\""
Jul 15 20:06:50.345009 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.344987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"]
Jul 15 20:06:50.385140 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.385102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:50.479210 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.479170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00087ad8-1653-496d-9d85-f71f84d2ad0b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.479444 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.479308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz6st\" (UniqueName: \"kubernetes.io/projected/00087ad8-1653-496d-9d85-f71f84d2ad0b-kube-api-access-lz6st\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.479444 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.479353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00087ad8-1653-496d-9d85-f71f84d2ad0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.580674 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.580580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz6st\" (UniqueName: \"kubernetes.io/projected/00087ad8-1653-496d-9d85-f71f84d2ad0b-kube-api-access-lz6st\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.580674 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.580628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00087ad8-1653-496d-9d85-f71f84d2ad0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.580872 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.580684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00087ad8-1653-496d-9d85-f71f84d2ad0b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.581319 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.581287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00087ad8-1653-496d-9d85-f71f84d2ad0b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.582816 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.582794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:50.583489 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.583469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00087ad8-1653-496d-9d85-f71f84d2ad0b-maas-api-tls\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.588086 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.588059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz6st\" (UniqueName: \"kubernetes.io/projected/00087ad8-1653-496d-9d85-f71f84d2ad0b-kube-api-access-lz6st\") pod \"maas-api-e2e-inf-a-a86716-685f5965b5-qsmps\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.647892 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.647864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:50.789143 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.789112    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"]
Jul 15 20:06:50.895463 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.895343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:50.999940 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:50.999890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:51.072180 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:51.072146    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps" event={"ID":"00087ad8-1653-496d-9d85-f71f84d2ad0b","Type":"ContainerStarted","Data":"ceb6107b021a293a3c3d58dacf5a885972e49c31c7e135a5c618e8c718b5bf6c"}
Jul 15 20:06:53.085403 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:53.085366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps" event={"ID":"00087ad8-1653-496d-9d85-f71f84d2ad0b","Type":"ContainerStarted","Data":"f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2"}
Jul 15 20:06:53.085783 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:53.085436    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:06:53.102797 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:53.102744    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps" podStartSLOduration=1.582458531 podStartE2EDuration="3.102730303s" totalImagesPullingTime="1.520271772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:06:53.099181696 +0000 UTC m=+2303.176974497" watchObservedRunningTime="2026-07-15 20:06:53.102730303 +0000 UTC m=+2303.180523105"
Jul 15 20:06:55.299254 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:55.299209    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:55.766690 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:55.766654    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:56.201276 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:56.201216    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:57.364968 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:57.364925    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:06:59.096710 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:06:59.096680    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:07:00.165107 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.165070    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"]
Jul 15 20:07:00.169682 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.169656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.171737 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.171711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c9860b-serving-cert\""
Jul 15 20:07:00.178562 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.178539    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"]
Jul 15 20:07:00.271506 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.271474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.271676 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.271547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.271676 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.271606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btkxq\" (UniqueName: \"kubernetes.io/projected/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-kube-api-access-btkxq\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.372488 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.372461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.372660 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.372540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.372660 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.372564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btkxq\" (UniqueName: \"kubernetes.io/projected/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-kube-api-access-btkxq\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.372660 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:07:00.372598    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-serving-cert: secret "maas-api-e2e-inf-b-c9860b-serving-cert" not found
Jul 15 20:07:00.372783 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:07:00.372662    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls podName:cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d nodeName:}" failed. No retries permitted until 2026-07-15 20:07:00.872645836 +0000 UTC m=+2310.950438616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls") pod "maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" (UID: "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d") : secret "maas-api-e2e-inf-b-c9860b-serving-cert" not found
Jul 15 20:07:00.373192 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.373174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.381795 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.381776    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btkxq\" (UniqueName: \"kubernetes.io/projected/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-kube-api-access-btkxq\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.876737 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.876700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.879439 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.879418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls\") pod \"maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:00.908793 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:00.908762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:01.010811 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:01.010760    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:01.083060 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:01.083024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:01.239123 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:01.239083    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"]
Jul 15 20:07:01.241639 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:07:01.241607    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbb56b0_5cc7_45fe_bb1c_86396dffcf0d.slice/crio-670ebb6b07936c8921d64406795006ca3a74b3bec2d237ef45078f840a9b4395 WatchSource:0}: Error finding container 670ebb6b07936c8921d64406795006ca3a74b3bec2d237ef45078f840a9b4395: Status 404 returned error can't find the container with id 670ebb6b07936c8921d64406795006ca3a74b3bec2d237ef45078f840a9b4395
Jul 15 20:07:02.135940 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:02.135905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" event={"ID":"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d","Type":"ContainerStarted","Data":"3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c"}
Jul 15 20:07:02.136138 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:02.135964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" event={"ID":"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d","Type":"ContainerStarted","Data":"670ebb6b07936c8921d64406795006ca3a74b3bec2d237ef45078f840a9b4395"}
Jul 15 20:07:02.136138 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:02.136025    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:02.152176 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:02.152130    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" podStartSLOduration=1.7526235190000001 podStartE2EDuration="2.152114284s" totalImagesPullingTime="399.490765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:07:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:07:02.15056397 +0000 UTC m=+2312.228356771" watchObservedRunningTime="2026-07-15 20:07:02.152114284 +0000 UTC m=+2312.229907084"
Jul 15 20:07:04.625664 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:04.625628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:08.147764 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:08.147735    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:07:11.666794 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:11.666747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:12.531353 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:12.531318    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:21.947155 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:21.947069    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:22.638920 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:22.638885    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:22.740463 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:22.740427    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:47.336638 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.336593    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"]
Jul 15 20:07:47.340389 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.340366    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.342470 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.342447    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c9860b\"/\"kube-root-ca.crt\""
Jul 15 20:07:47.343062 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.343020    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c9860b\"/\"default-dockercfg-x84j9\""
Jul 15 20:07:47.343203 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.343181    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c9860b\"/\"test-model-c9860b-kserve-self-signed-certs\""
Jul 15 20:07:47.343323 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.343304    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c9860b\"/\"openshift-service-ca.crt\""
Jul 15 20:07:47.350811 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.350786    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"]
Jul 15 20:07:47.431304 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.431273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:07:47.509042 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.509007    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbqkm\" (UniqueName: \"kubernetes.io/projected/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-kube-api-access-xbqkm\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.509187 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.509047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-dshm\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.509187 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.509161    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tmp-dir\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.509308 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.509267    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tls-certs\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.509308 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.509291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-model-cache\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.509386 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.509305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-home\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.610526 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tmp-dir\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.610692 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tls-certs\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.610692 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610578    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-model-cache\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.610818 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-home\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.610818 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbqkm\" (UniqueName: \"kubernetes.io/projected/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-kube-api-access-xbqkm\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.610818 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-dshm\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.611039 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tmp-dir\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.611039 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.610937    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-model-cache\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.611140 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.611081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-home\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.613104 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.613078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-dshm\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.613360 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.613344    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tls-certs\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.617776 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.617758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbqkm\" (UniqueName: \"kubernetes.io/projected/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-kube-api-access-xbqkm\") pod \"test-model-c9860b-kserve-76968bfcf7-vv2w4\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") " pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.651460 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.651432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:47.787874 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:47.787843    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"]
Jul 15 20:07:47.789358 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:07:47.789326    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfff4165_78f9_48c8_b3f1_cc896d6e96e2.slice/crio-d42796c11ecef97045e3a9829f37b7c5431171668b8aedf4694711c8b1c04c08 WatchSource:0}: Error finding container d42796c11ecef97045e3a9829f37b7c5431171668b8aedf4694711c8b1c04c08: Status 404 returned error can't find the container with id d42796c11ecef97045e3a9829f37b7c5431171668b8aedf4694711c8b1c04c08
Jul 15 20:07:48.414129 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:48.414093    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4" event={"ID":"cfff4165-78f9-48c8-b3f1-cc896d6e96e2","Type":"ContainerStarted","Data":"9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51"}
Jul 15 20:07:48.414129 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:48.414138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4" event={"ID":"cfff4165-78f9-48c8-b3f1-cc896d6e96e2","Type":"ContainerStarted","Data":"d42796c11ecef97045e3a9829f37b7c5431171668b8aedf4694711c8b1c04c08"}
Jul 15 20:07:48.430318 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:48.430268    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4" podStartSLOduration=1.430253353 podStartE2EDuration="1.430253353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:07:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:07:48.428152304 +0000 UTC m=+2358.505945105" watchObservedRunningTime="2026-07-15 20:07:48.430253353 +0000 UTC m=+2358.508046309"
Jul 15 20:07:57.651606 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:57.651569    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:57.652055 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:57.651638    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:57.665272 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:57.664886    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:07:58.482201 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:07:58.482171    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:08:13.942151 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:13.942110    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:14.884440 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:14.884407    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"]
Jul 15 20:08:14.893759 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:14.893694    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:15.595069 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:15.595037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:15.698379 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:15.698349    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:25.460114 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:25.460077    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"]
Jul 15 20:08:25.460659 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:25.460414    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps" podUID="00087ad8-1653-496d-9d85-f71f84d2ad0b" containerName="maas-api" containerID="cri-o://f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2" gracePeriod=30
Jul 15 20:08:25.489345 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:25.489313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:25.910926 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:25.910894    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:08:26.069458 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.069359    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00087ad8-1653-496d-9d85-f71f84d2ad0b-openshift-service-ca\") pod \"00087ad8-1653-496d-9d85-f71f84d2ad0b\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") "
Jul 15 20:08:26.069458 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.069449    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz6st\" (UniqueName: \"kubernetes.io/projected/00087ad8-1653-496d-9d85-f71f84d2ad0b-kube-api-access-lz6st\") pod \"00087ad8-1653-496d-9d85-f71f84d2ad0b\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") "
Jul 15 20:08:26.069680 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.069497    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00087ad8-1653-496d-9d85-f71f84d2ad0b-maas-api-tls\") pod \"00087ad8-1653-496d-9d85-f71f84d2ad0b\" (UID: \"00087ad8-1653-496d-9d85-f71f84d2ad0b\") "
Jul 15 20:08:26.069827 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.069797    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00087ad8-1653-496d-9d85-f71f84d2ad0b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00087ad8-1653-496d-9d85-f71f84d2ad0b" (UID: "00087ad8-1653-496d-9d85-f71f84d2ad0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:08:26.071794 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.071759    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00087ad8-1653-496d-9d85-f71f84d2ad0b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00087ad8-1653-496d-9d85-f71f84d2ad0b" (UID: "00087ad8-1653-496d-9d85-f71f84d2ad0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:08:26.071794 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.071763    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00087ad8-1653-496d-9d85-f71f84d2ad0b-kube-api-access-lz6st" (OuterVolumeSpecName: "kube-api-access-lz6st") pod "00087ad8-1653-496d-9d85-f71f84d2ad0b" (UID: "00087ad8-1653-496d-9d85-f71f84d2ad0b"). InnerVolumeSpecName "kube-api-access-lz6st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:08:26.171293 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.171259    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00087ad8-1653-496d-9d85-f71f84d2ad0b-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:26.171293 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.171291    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz6st\" (UniqueName: \"kubernetes.io/projected/00087ad8-1653-496d-9d85-f71f84d2ad0b-kube-api-access-lz6st\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:26.171491 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.171307    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00087ad8-1653-496d-9d85-f71f84d2ad0b-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:26.641333 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.641291    2564 generic.go:356] "Generic (PLEG): container finished" podID="00087ad8-1653-496d-9d85-f71f84d2ad0b" containerID="f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2" exitCode=0
Jul 15 20:08:26.641333 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.641329    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"
Jul 15 20:08:26.641831 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.641344    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps" event={"ID":"00087ad8-1653-496d-9d85-f71f84d2ad0b","Type":"ContainerDied","Data":"f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2"}
Jul 15 20:08:26.641831 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.641397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps" event={"ID":"00087ad8-1653-496d-9d85-f71f84d2ad0b","Type":"ContainerDied","Data":"ceb6107b021a293a3c3d58dacf5a885972e49c31c7e135a5c618e8c718b5bf6c"}
Jul 15 20:08:26.641831 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.641421    2564 scope.go:117] "RemoveContainer" containerID="f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2"
Jul 15 20:08:26.651136 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.651114    2564 scope.go:117] "RemoveContainer" containerID="f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2"
Jul 15 20:08:26.651439 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:08:26.651421    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2\": container with ID starting with f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2 not found: ID does not exist" containerID="f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2"
Jul 15 20:08:26.651517 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.651449    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2"} err="failed to get container status \"f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2\": rpc error: code = NotFound desc = could not find container \"f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2\": container with ID starting with f0e0146f28c4c3cb395fd3b9ca400cc0d65caf8dfeea81d7135b4e136e5822e2 not found: ID does not exist"
Jul 15 20:08:26.658835 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.658813    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"]
Jul 15 20:08:26.667609 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:26.667588    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a86716-685f5965b5-qsmps"]
Jul 15 20:08:28.493184 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:28.493147    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00087ad8-1653-496d-9d85-f71f84d2ad0b" path="/var/lib/kubelet/pods/00087ad8-1653-496d-9d85-f71f84d2ad0b/volumes"
Jul 15 20:08:29.905361 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:29.905320    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4" podUID="cfff4165-78f9-48c8-b3f1-cc896d6e96e2" containerName="main" containerID="cri-o://9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51" gracePeriod=45
Jul 15 20:08:30.145612 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.145583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:30.159675 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.159622    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:08:30.308023 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.307996    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-model-cache\") pod \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") "
Jul 15 20:08:30.308253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308032    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbqkm\" (UniqueName: \"kubernetes.io/projected/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-kube-api-access-xbqkm\") pod \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") "
Jul 15 20:08:30.308253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308058    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tmp-dir\") pod \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") "
Jul 15 20:08:30.308253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308078    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-home\") pod \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") "
Jul 15 20:08:30.308253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308096    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-dshm\") pod \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") "
Jul 15 20:08:30.308253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308178    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tls-certs\") pod \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\" (UID: \"cfff4165-78f9-48c8-b3f1-cc896d6e96e2\") "
Jul 15 20:08:30.308525 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308368    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfff4165-78f9-48c8-b3f1-cc896d6e96e2" (UID: "cfff4165-78f9-48c8-b3f1-cc896d6e96e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:08:30.308525 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308366    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-home" (OuterVolumeSpecName: "home") pod "cfff4165-78f9-48c8-b3f1-cc896d6e96e2" (UID: "cfff4165-78f9-48c8-b3f1-cc896d6e96e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:08:30.308525 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308385    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfff4165-78f9-48c8-b3f1-cc896d6e96e2" (UID: "cfff4165-78f9-48c8-b3f1-cc896d6e96e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:08:30.308525 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308479    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:30.308525 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308524    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:30.308942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.308539    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:30.310488 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.310456    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-kube-api-access-xbqkm" (OuterVolumeSpecName: "kube-api-access-xbqkm") pod "cfff4165-78f9-48c8-b3f1-cc896d6e96e2" (UID: "cfff4165-78f9-48c8-b3f1-cc896d6e96e2"). InnerVolumeSpecName "kube-api-access-xbqkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:08:30.310618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.310512    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfff4165-78f9-48c8-b3f1-cc896d6e96e2" (UID: "cfff4165-78f9-48c8-b3f1-cc896d6e96e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:08:30.310618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.310545    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-dshm" (OuterVolumeSpecName: "dshm") pod "cfff4165-78f9-48c8-b3f1-cc896d6e96e2" (UID: "cfff4165-78f9-48c8-b3f1-cc896d6e96e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:08:30.409118 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.409080    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbqkm\" (UniqueName: \"kubernetes.io/projected/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-kube-api-access-xbqkm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:30.409118 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.409110    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:30.409118 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.409122    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfff4165-78f9-48c8-b3f1-cc896d6e96e2-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:30.542342 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.542309    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:30.665005 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.664975    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 20:08:30.668996 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.668966    2564 generic.go:356] "Generic (PLEG): container finished" podID="cfff4165-78f9-48c8-b3f1-cc896d6e96e2" containerID="9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51" exitCode=0
Jul 15 20:08:30.669169 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.669051    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4" event={"ID":"cfff4165-78f9-48c8-b3f1-cc896d6e96e2","Type":"ContainerDied","Data":"9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51"}
Jul 15 20:08:30.669169 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.669080    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"
Jul 15 20:08:30.669169 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.669091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4" event={"ID":"cfff4165-78f9-48c8-b3f1-cc896d6e96e2","Type":"ContainerDied","Data":"d42796c11ecef97045e3a9829f37b7c5431171668b8aedf4694711c8b1c04c08"}
Jul 15 20:08:30.669169 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.669117    2564 scope.go:117] "RemoveContainer" containerID="9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51"
Jul 15 20:08:30.672642 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.672623    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 20:08:30.679061 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.679040    2564 scope.go:117] "RemoveContainer" containerID="9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51"
Jul 15 20:08:30.679324 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:08:30.679300    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51\": container with ID starting with 9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51 not found: ID does not exist" containerID="9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51"
Jul 15 20:08:30.679384 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.679333    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51"} err="failed to get container status \"9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51\": rpc error: code = NotFound desc = could not find container \"9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51\": container with ID starting with 9e8a6b4b5de972a26815f76c89d99370b9df38b033c880848e9c243ca32a0f51 not found: ID does not exist"
Jul 15 20:08:30.685354 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.685332    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"]
Jul 15 20:08:30.696139 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:30.696116    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c9860b/test-model-c9860b-kserve-76968bfcf7-vv2w4"]
Jul 15 20:08:32.492928 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:32.492883    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfff4165-78f9-48c8-b3f1-cc896d6e96e2" path="/var/lib/kubelet/pods/cfff4165-78f9-48c8-b3f1-cc896d6e96e2/volumes"
Jul 15 20:08:36.364141 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:36.364104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:36.423469 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:36.423439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:36.671708 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:36.671621    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:36.917579 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:36.917542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:37.034925 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.034879    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"]
Jul 15 20:08:37.035406 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.035387    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfff4165-78f9-48c8-b3f1-cc896d6e96e2" containerName="main"
Jul 15 20:08:37.035406 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.035407    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfff4165-78f9-48c8-b3f1-cc896d6e96e2" containerName="main"
Jul 15 20:08:37.035550 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.035423    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00087ad8-1653-496d-9d85-f71f84d2ad0b" containerName="maas-api"
Jul 15 20:08:37.035550 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.035429    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="00087ad8-1653-496d-9d85-f71f84d2ad0b" containerName="maas-api"
Jul 15 20:08:37.035550 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.035509    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="00087ad8-1653-496d-9d85-f71f84d2ad0b" containerName="maas-api"
Jul 15 20:08:37.035550 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.035521    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfff4165-78f9-48c8-b3f1-cc896d6e96e2" containerName="main"
Jul 15 20:08:37.038852 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.038829    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:37.041462 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.041439    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:08:37.044718 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.044697    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"]
Jul 15 20:08:37.075098 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.075060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b513-036c-4f41-9c35-72d93f860023-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2\" (UID: \"2ef7b513-036c-4f41-9c35-72d93f860023\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:37.175888 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.175852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b513-036c-4f41-9c35-72d93f860023-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2\" (UID: \"2ef7b513-036c-4f41-9c35-72d93f860023\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:37.176585 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.176565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b513-036c-4f41-9c35-72d93f860023-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2\" (UID: \"2ef7b513-036c-4f41-9c35-72d93f860023\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:37.352121 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.352031    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:37.485870 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.485843    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"]
Jul 15 20:08:37.487479 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:08:37.487445    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef7b513_036c_4f41_9c35_72d93f860023.slice/crio-6bff9f8d5ebe272aef810605f3a06eae6983e49546973e5299ce3185775ae9b9 WatchSource:0}: Error finding container 6bff9f8d5ebe272aef810605f3a06eae6983e49546973e5299ce3185775ae9b9: Status 404 returned error can't find the container with id 6bff9f8d5ebe272aef810605f3a06eae6983e49546973e5299ce3185775ae9b9
Jul 15 20:08:37.686462 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.686426    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:37.712378 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.712343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2" event={"ID":"2ef7b513-036c-4f41-9c35-72d93f860023","Type":"ContainerStarted","Data":"6bff9f8d5ebe272aef810605f3a06eae6983e49546973e5299ce3185775ae9b9"}
Jul 15 20:08:37.760112 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:37.760079    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:38.719983 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:38.719948    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ef7b513-036c-4f41-9c35-72d93f860023" containerID="98b1059a57503e4255aef324c5ee02086ca73ba3ee6c8c3145e80e891ad46b6c" exitCode=0
Jul 15 20:08:38.720378 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:38.720030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2" event={"ID":"2ef7b513-036c-4f41-9c35-72d93f860023","Type":"ContainerDied","Data":"98b1059a57503e4255aef324c5ee02086ca73ba3ee6c8c3145e80e891ad46b6c"}
Jul 15 20:08:39.864086 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:39.864056    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:39.901662 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:39.901630    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b513-036c-4f41-9c35-72d93f860023-maas-api-service-ca\") pod \"2ef7b513-036c-4f41-9c35-72d93f860023\" (UID: \"2ef7b513-036c-4f41-9c35-72d93f860023\") "
Jul 15 20:08:39.901975 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:39.901952    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ef7b513-036c-4f41-9c35-72d93f860023-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ef7b513-036c-4f41-9c35-72d93f860023" (UID: "2ef7b513-036c-4f41-9c35-72d93f860023"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:08:40.002733 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:40.002642    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b513-036c-4f41-9c35-72d93f860023-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:40.733989 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:40.733940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2" event={"ID":"2ef7b513-036c-4f41-9c35-72d93f860023","Type":"ContainerDied","Data":"6bff9f8d5ebe272aef810605f3a06eae6983e49546973e5299ce3185775ae9b9"}
Jul 15 20:08:40.734191 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:40.734053    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bff9f8d5ebe272aef810605f3a06eae6983e49546973e5299ce3185775ae9b9"
Jul 15 20:08:40.734191 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:40.734132    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2"
Jul 15 20:08:43.849723 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:43.849687    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:44.578832 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:44.578802    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:47.709654 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:47.709619    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"]
Jul 15 20:08:47.710033 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:47.709867    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" podUID="cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" containerName="maas-api" containerID="cri-o://3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c" gracePeriod=30
Jul 15 20:08:47.966574 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:47.966509    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:08:48.077858 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.077824    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-openshift-service-ca\") pod \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") "
Jul 15 20:08:48.078032 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.077873    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btkxq\" (UniqueName: \"kubernetes.io/projected/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-kube-api-access-btkxq\") pod \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") "
Jul 15 20:08:48.078032 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.077927    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls\") pod \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\" (UID: \"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d\") "
Jul 15 20:08:48.078289 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.078260    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" (UID: "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:08:48.080134 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.080101    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" (UID: "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:08:48.080263 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.080170    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-kube-api-access-btkxq" (OuterVolumeSpecName: "kube-api-access-btkxq") pod "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" (UID: "cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d"). InnerVolumeSpecName "kube-api-access-btkxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:08:48.179450 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.179412    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:48.179450 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.179442    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:48.179450 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.179452    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btkxq\" (UniqueName: \"kubernetes.io/projected/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d-kube-api-access-btkxq\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:08:48.781295 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.781193    2564 generic.go:356] "Generic (PLEG): container finished" podID="cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" containerID="3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c" exitCode=0
Jul 15 20:08:48.781295 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.781257    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"
Jul 15 20:08:48.781295 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.781284    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" event={"ID":"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d","Type":"ContainerDied","Data":"3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c"}
Jul 15 20:08:48.781784 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.781341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g" event={"ID":"cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d","Type":"ContainerDied","Data":"670ebb6b07936c8921d64406795006ca3a74b3bec2d237ef45078f840a9b4395"}
Jul 15 20:08:48.781784 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.781364    2564 scope.go:117] "RemoveContainer" containerID="3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c"
Jul 15 20:08:48.790808 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.790781    2564 scope.go:117] "RemoveContainer" containerID="3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c"
Jul 15 20:08:48.791064 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:08:48.791044    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c\": container with ID starting with 3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c not found: ID does not exist" containerID="3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c"
Jul 15 20:08:48.791122 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.791075    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c"} err="failed to get container status \"3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c\": rpc error: code = NotFound desc = could not find container \"3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c\": container with ID starting with 3a113f5ecec2e2d6718c6f903b12dfdaf4714b1fa7b35a04052ebd6649102c6c not found: ID does not exist"
Jul 15 20:08:48.799920 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.799897    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"]
Jul 15 20:08:48.804432 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.804411    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c9860b-68fdb95c99-k4m6g"]
Jul 15 20:08:48.916406 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:48.916372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:50.494647 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:50.494606    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" path="/var/lib/kubelet/pods/cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d/volumes"
Jul 15 20:08:52.225192 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:52.225116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:52.641215 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:52.641134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:58.381415 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:58.381353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:58.438405 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:58.438369    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:58.671561 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:58.671482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:58.963187 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:58.963120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:59.308455 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:59.308368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:08:59.629049 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:08:59.628971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:00.267211 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:00.267176    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:02.283317 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.283284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:02.365093 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.365054    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:02.372820 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.372792    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"]
Jul 15 20:09:02.373432 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.373416    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" containerName="maas-api"
Jul 15 20:09:02.373480 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.373434    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" containerName="maas-api"
Jul 15 20:09:02.373480 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.373474    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef7b513-036c-4f41-9c35-72d93f860023" containerName="revoke-keys"
Jul 15 20:09:02.373544 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.373484    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef7b513-036c-4f41-9c35-72d93f860023" containerName="revoke-keys"
Jul 15 20:09:02.373580 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.373552    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdbb56b0-5cc7-45fe-bb1c-86396dffcf0d" containerName="maas-api"
Jul 15 20:09:02.373580 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.373564    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef7b513-036c-4f41-9c35-72d93f860023" containerName="revoke-keys"
Jul 15 20:09:02.378121 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.378103    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.380417 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.380396    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fc7b22-serving-cert\""
Jul 15 20:09:02.387201 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.387174    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"]
Jul 15 20:09:02.458450 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.458418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:02.507013 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.506986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-maas-api-tls\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.507245 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.507064    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-openshift-service-ca\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.507245 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.507134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c48v9\" (UniqueName: \"kubernetes.io/projected/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-kube-api-access-c48v9\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.558264 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.558163    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:02.608156 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.608121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-openshift-service-ca\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.608376 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.608183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c48v9\" (UniqueName: \"kubernetes.io/projected/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-kube-api-access-c48v9\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.608376 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.608264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-maas-api-tls\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.608827 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.608802    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-openshift-service-ca\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.610814 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.610795    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-maas-api-tls\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.616309 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.616286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c48v9\" (UniqueName: \"kubernetes.io/projected/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-kube-api-access-c48v9\") pod \"maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.689977 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.689941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:02.829048 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.829018    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"]
Jul 15 20:09:02.830717 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:09:02.830689    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8e7b8e_be3f_44dd_8d8a_5ed608ef1ab1.slice/crio-4dc98d892cfa00f421d13dd95112bc6518235a3302a7ed813f9ea0aa821a2023 WatchSource:0}: Error finding container 4dc98d892cfa00f421d13dd95112bc6518235a3302a7ed813f9ea0aa821a2023: Status 404 returned error can't find the container with id 4dc98d892cfa00f421d13dd95112bc6518235a3302a7ed813f9ea0aa821a2023
Jul 15 20:09:02.831906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.831886    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:09:02.869695 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.869661    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x" event={"ID":"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1","Type":"ContainerStarted","Data":"4dc98d892cfa00f421d13dd95112bc6518235a3302a7ed813f9ea0aa821a2023"}
Jul 15 20:09:02.965462 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:02.965427    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:03.069275 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:03.069208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:03.877940 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:03.877853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x" event={"ID":"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1","Type":"ContainerStarted","Data":"52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0"}
Jul 15 20:09:03.878296 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:03.878044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:03.895444 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:03.895396    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x" podStartSLOduration=1.25258981 podStartE2EDuration="1.895381179s" totalImagesPullingTime="642.791369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:09:03.892028642 +0000 UTC m=+2433.969821442" watchObservedRunningTime="2026-07-15 20:09:03.895381179 +0000 UTC m=+2433.973173979"
Jul 15 20:09:05.035239 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:05.035167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:05.150404 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:05.150369    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:05.284521 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:05.284488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:05.854328 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:05.854295    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:06.821486 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:06.821446    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:07.306826 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.306789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:07.610408 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.610323    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"]
Jul 15 20:09:07.613882 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.613860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.616340 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.616317    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-5306d0-openshift-default-dockercfg-9rdm5\""
Jul 15 20:09:07.616462 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.616367    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 20:09:07.623983 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.623958    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"]
Jul 15 20:09:07.635735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.635691    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:07.758164 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b548h\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-kube-api-access-b548h\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-data\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-credential-socket\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758273    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-token\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-envoy\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758331    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-socket\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758535 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istiod-ca-cert\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758535 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-certs\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.758535 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.758465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-podinfo\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.798649 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.798604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:07.859693 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-socket\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.859693 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859695    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istiod-ca-cert\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-certs\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-podinfo\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b548h\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-kube-api-access-b548h\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859821    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-data\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-credential-socket\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.859909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-token\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.860001    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-envoy\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860175 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.860120    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-socket\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860659 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.860189    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-certs\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860659 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.860471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-data\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860659 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.860489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istiod-ca-cert\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.860659 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.860620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-credential-socket\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.862362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.862341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-envoy\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.862519 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.862502    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-podinfo\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.868154 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.868128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-token\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.868305 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.868197    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b548h\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-kube-api-access-b548h\") pod \"e2e-api-b-5306d0-openshift-default-78fd788989-s296t\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") " pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:07.926811 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:07.926777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:08.064348 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:08.064319    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"]
Jul 15 20:09:08.065480 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:09:08.065455    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7881ac1b_8b45_421a_8228_0fe03ed10c2a.slice/crio-f91e754ea1df7d0fd7de7d9faa84f2aaf38848fbb9de0f82545c5efe5730d975 WatchSource:0}: Error finding container f91e754ea1df7d0fd7de7d9faa84f2aaf38848fbb9de0f82545c5efe5730d975: Status 404 returned error can't find the container with id f91e754ea1df7d0fd7de7d9faa84f2aaf38848fbb9de0f82545c5efe5730d975
Jul 15 20:09:08.909997 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:08.909959    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" event={"ID":"7881ac1b-8b45-421a-8228-0fe03ed10c2a","Type":"ContainerStarted","Data":"f91e754ea1df7d0fd7de7d9faa84f2aaf38848fbb9de0f82545c5efe5730d975"}
Jul 15 20:09:09.890789 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:09.890756    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:10.691295 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:10.691252    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 20:09:10.691687 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:10.691350    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 20:09:10.691687 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:10.691394    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 20:09:10.924710 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:10.924673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" event={"ID":"7881ac1b-8b45-421a-8228-0fe03ed10c2a","Type":"ContainerStarted","Data":"699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"}
Jul 15 20:09:10.927304 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:10.927265    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:10.947630 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:10.947517    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" podStartSLOduration=1.324235391 podStartE2EDuration="3.947493638s" totalImagesPullingTime="2.623258247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:09:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:09:10.941793721 +0000 UTC m=+2441.019586525" watchObservedRunningTime="2026-07-15 20:09:10.947493638 +0000 UTC m=+2441.025286441"
Jul 15 20:09:11.932514 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:11.932484    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:12.908621 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:12.908579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:12.939379 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:12.939347    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:12.940636 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:12.940613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:13.012757 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:13.012722    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:13.133427 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:13.133385    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:13.441684 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:13.441650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:13.697608 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:13.697521    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:13.797969 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:13.797931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:17.437086 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:17.437054    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:23.999754 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:23.999714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:24.102016 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:24.101971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:27.021827 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.021796    2564 generic.go:356] "Generic (PLEG): container finished" podID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerID="699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9" exitCode=0
Jul 15 20:09:27.022178 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.021872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" event={"ID":"7881ac1b-8b45-421a-8228-0fe03ed10c2a","Type":"ContainerDied","Data":"699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"}
Jul 15 20:09:27.022340 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.022324    2564 scope.go:117] "RemoveContainer" containerID="699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"
Jul 15 20:09:27.024261 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.024211    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 20:09:27.024373 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.024319    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 20:09:27.024436 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.024369    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 20:09:27.941673 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:27.941486    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:28.029794 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:28.029754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" event={"ID":"7881ac1b-8b45-421a-8228-0fe03ed10c2a","Type":"ContainerStarted","Data":"d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1"}
Jul 15 20:09:28.927820 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:28.927789    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:28.929016 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:28.928989    2564 patch_prober.go:28] interesting pod/e2e-api-b-5306d0-openshift-default-78fd788989-s296t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 15 20:09:28.929152 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:28.929039    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 15 20:09:29.928055 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:29.928019    2564 patch_prober.go:28] interesting pod/e2e-api-b-5306d0-openshift-default-78fd788989-s296t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 15 20:09:29.928474 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:29.928076    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 15 20:09:30.927682 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:30.927647    2564 patch_prober.go:28] interesting pod/e2e-api-b-5306d0-openshift-default-78fd788989-s296t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 15 20:09:30.927850 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:30.927697    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 15 20:09:31.927709 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:31.927672    2564 patch_prober.go:28] interesting pod/e2e-api-b-5306d0-openshift-default-78fd788989-s296t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 15 20:09:31.928102 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:31.927727    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 15 20:09:32.931584 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:32.931549    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:32.931961 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:32.931925    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:32.932690 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:32.932671    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:09:34.476960 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:34.476924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:34.578478 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:34.578444    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:36.165873 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:36.165832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:36.265307 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:36.265275    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:41.403457 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.403423    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"]
Jul 15 20:09:41.403833 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.403639    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x" podUID="8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" containerName="maas-api" containerID="cri-o://52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0" gracePeriod=30
Jul 15 20:09:41.412185 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.412161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:41.650401 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.650377    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:41.791094 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.791063    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-openshift-service-ca\") pod \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") "
Jul 15 20:09:41.791301 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.791157    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-maas-api-tls\") pod \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") "
Jul 15 20:09:41.791301 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.791192    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c48v9\" (UniqueName: \"kubernetes.io/projected/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-kube-api-access-c48v9\") pod \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\" (UID: \"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1\") "
Jul 15 20:09:41.791504 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.791478    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" (UID: "8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:09:41.791580 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.791563    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:09:41.793512 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.793489    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" (UID: "8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:09:41.793512 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.793501    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-kube-api-access-c48v9" (OuterVolumeSpecName: "kube-api-access-c48v9") pod "8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" (UID: "8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1"). InnerVolumeSpecName "kube-api-access-c48v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:09:41.892942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.892902    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:09:41.892942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:41.892938    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c48v9\" (UniqueName: \"kubernetes.io/projected/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1-kube-api-access-c48v9\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:09:42.123980 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.123892    2564 generic.go:356] "Generic (PLEG): container finished" podID="8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" containerID="52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0" exitCode=0
Jul 15 20:09:42.123980 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.123939    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"
Jul 15 20:09:42.124164 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.123974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x" event={"ID":"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1","Type":"ContainerDied","Data":"52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0"}
Jul 15 20:09:42.124164 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.124027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x" event={"ID":"8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1","Type":"ContainerDied","Data":"4dc98d892cfa00f421d13dd95112bc6518235a3302a7ed813f9ea0aa821a2023"}
Jul 15 20:09:42.124164 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.124050    2564 scope.go:117] "RemoveContainer" containerID="52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0"
Jul 15 20:09:42.135452 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.135277    2564 scope.go:117] "RemoveContainer" containerID="52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0"
Jul 15 20:09:42.135761 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:09:42.135742    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0\": container with ID starting with 52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0 not found: ID does not exist" containerID="52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0"
Jul 15 20:09:42.135809 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.135770    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0"} err="failed to get container status \"52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0\": rpc error: code = NotFound desc = could not find container \"52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0\": container with ID starting with 52e8ea278e666a23588cfa7d1ceb7bce54a27e0cd12a3139dad97d17b4c25af0 not found: ID does not exist"
Jul 15 20:09:42.146602 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.146577    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"]
Jul 15 20:09:42.150837 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.150815    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc7b22-667fbc6f59-5lj6x"]
Jul 15 20:09:42.493418 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:42.493375    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" path="/var/lib/kubelet/pods/8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1/volumes"
Jul 15 20:09:45.690372 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:45.690336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:46.091481 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:46.091443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:51.935566 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:51.935530    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:52.038560 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:52.038498    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:52.247669 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:52.247633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:52.512346 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:52.512214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:53.351390 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:53.351339    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:09:53.489198 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:09:53.489141    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:08.077801 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:08.077746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:10.192971 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:10.192894    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:12.719681 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:12.719639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:13.142339 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:13.142211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:18.679238 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:18.679162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:18.736438 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:18.736406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:19.015935 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:19.015894    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"]
Jul 15 20:10:19.016147 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:19.016120    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy" containerID="cri-o://d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1" gracePeriod=30
Jul 15 20:10:19.064347 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:19.064313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:19.252275 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:19.252238    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:19.610467 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:19.610398    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:19.938124 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:19.938023    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:20.133318 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:20.133280    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:22.814657 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:22.814561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:22.915339 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:22.915304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:23.085862 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:23.085767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:23.409885 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:23.409801    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:23.511729 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:23.511680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:24.282936 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.282875    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:10:24.373352 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.373322    2564 generic.go:356] "Generic (PLEG): container finished" podID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerID="d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1" exitCode=0
Jul 15 20:10:24.373534 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.373374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" event={"ID":"7881ac1b-8b45-421a-8228-0fe03ed10c2a","Type":"ContainerDied","Data":"d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1"}
Jul 15 20:10:24.373534 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.373378    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"
Jul 15 20:10:24.373534 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.373416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t" event={"ID":"7881ac1b-8b45-421a-8228-0fe03ed10c2a","Type":"ContainerDied","Data":"f91e754ea1df7d0fd7de7d9faa84f2aaf38848fbb9de0f82545c5efe5730d975"}
Jul 15 20:10:24.373534 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.373439    2564 scope.go:117] "RemoveContainer" containerID="d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1"
Jul 15 20:10:24.382870 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.382851    2564 scope.go:117] "RemoveContainer" containerID="699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"
Jul 15 20:10:24.391287 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.391267    2564 scope.go:117] "RemoveContainer" containerID="d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1"
Jul 15 20:10:24.391541 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:10:24.391520    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1\": container with ID starting with d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1 not found: ID does not exist" containerID="d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1"
Jul 15 20:10:24.391617 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.391554    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1"} err="failed to get container status \"d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1\": rpc error: code = NotFound desc = could not find container \"d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1\": container with ID starting with d4bd8639cd44465ee6ad482d9623dff2590508a1d50f7465e6dbf4bdec634ef1 not found: ID does not exist"
Jul 15 20:10:24.391617 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.391580    2564 scope.go:117] "RemoveContainer" containerID="699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"
Jul 15 20:10:24.391835 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:10:24.391814    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9\": container with ID starting with 699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9 not found: ID does not exist" containerID="699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"
Jul 15 20:10:24.391878 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.391846    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9"} err="failed to get container status \"699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9\": rpc error: code = NotFound desc = could not find container \"699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9\": container with ID starting with 699bdd7548da676766e356d88aeee63327c0796d9205a82b3dd0210168bdc9e9 not found: ID does not exist"
Jul 15 20:10:24.396249 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396211    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-certs\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396347 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396273    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-socket\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396347 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396324    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b548h\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-kube-api-access-b548h\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396449 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396358    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-podinfo\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396449 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396424    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-data\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396563 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396472    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:10:24.396563 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396485    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istiod-ca-cert\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396563 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396497    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:10:24.396563 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396533    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-envoy\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396761 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396569    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-token\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.396761 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396598    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-credential-socket\") pod \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\" (UID: \"7881ac1b-8b45-421a-8228-0fe03ed10c2a\") "
Jul 15 20:10:24.397300 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396928    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.397300 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.397258    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.397300 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.396954    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:10:24.397300 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.397162    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:10:24.398160 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.398132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:10:24.399982 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.399955    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-kube-api-access-b548h" (OuterVolumeSpecName: "kube-api-access-b548h") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "kube-api-access-b548h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:10:24.399982 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.399967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:10:24.399982 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.399976    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:10:24.400209 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.400010    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7881ac1b-8b45-421a-8228-0fe03ed10c2a" (UID: "7881ac1b-8b45-421a-8228-0fe03ed10c2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 20:10:24.498676 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498644    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b548h\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-kube-api-access-b548h\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.498676 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498671    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.498676 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498682    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.498676 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498689    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.499049 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498698    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.499049 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498707    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7881ac1b-8b45-421a-8228-0fe03ed10c2a-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.499049 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.498714    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7881ac1b-8b45-421a-8228-0fe03ed10c2a-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:10:24.694544 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.694468    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"]
Jul 15 20:10:24.697210 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:24.697186    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-5306d0-openshift-default-78fd788989-s296t"]
Jul 15 20:10:26.492967 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:26.492932    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" path="/var/lib/kubelet/pods/7881ac1b-8b45-421a-8228-0fe03ed10c2a/volumes"
Jul 15 20:10:27.256577 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:27.256543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:27.975549 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:27.975507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:28.075981 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:28.075949    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:28.276249 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:28.276138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:30.848658 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.848619    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:30.925651 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.925601    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"]
Jul 15 20:10:30.926306 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.926283    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy"
Jul 15 20:10:30.926306 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.926306    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy"
Jul 15 20:10:30.926497 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.926329    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" containerName="maas-api"
Jul 15 20:10:30.926497 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.926337    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" containerName="maas-api"
Jul 15 20:10:30.926497 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.926454    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy"
Jul 15 20:10:30.926497 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.926475    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8e7b8e-be3f-44dd-8d8a-5ed608ef1ab1" containerName="maas-api"
Jul 15 20:10:30.928083 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.928044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:30.931234 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.931194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:30.933163 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.933137    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b72424-serving-cert\""
Jul 15 20:10:30.940084 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.939748    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"]
Jul 15 20:10:30.994871 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:30.994838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:31.061357 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.061312    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq5nj\" (UniqueName: \"kubernetes.io/projected/80ad1e76-2889-4a8a-9654-2f815582ac3e-kube-api-access-vq5nj\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.061561 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.061443    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ad1e76-2889-4a8a-9654-2f815582ac3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.061561 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.061489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ad1e76-2889-4a8a-9654-2f815582ac3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.162736 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.162646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ad1e76-2889-4a8a-9654-2f815582ac3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.162736 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.162698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq5nj\" (UniqueName: \"kubernetes.io/projected/80ad1e76-2889-4a8a-9654-2f815582ac3e-kube-api-access-vq5nj\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.162968 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.162759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ad1e76-2889-4a8a-9654-2f815582ac3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.163363 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.163342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ad1e76-2889-4a8a-9654-2f815582ac3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.165449 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.165424    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ad1e76-2889-4a8a-9654-2f815582ac3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.171048 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.171023    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq5nj\" (UniqueName: \"kubernetes.io/projected/80ad1e76-2889-4a8a-9654-2f815582ac3e-kube-api-access-vq5nj\") pod \"maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.194899 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.194869    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:31.243745 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.243709    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:31.383609 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.383583    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"]
Jul 15 20:10:31.384448 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:10:31.384420    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ad1e76_2889_4a8a_9654_2f815582ac3e.slice/crio-27407fb1ad37cf965bcde1737e07a7b6df52496caef9638763c73f7b626c1097 WatchSource:0}: Error finding container 27407fb1ad37cf965bcde1737e07a7b6df52496caef9638763c73f7b626c1097: Status 404 returned error can't find the container with id 27407fb1ad37cf965bcde1737e07a7b6df52496caef9638763c73f7b626c1097
Jul 15 20:10:31.415942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.415871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k" event={"ID":"80ad1e76-2889-4a8a-9654-2f815582ac3e","Type":"ContainerStarted","Data":"27407fb1ad37cf965bcde1737e07a7b6df52496caef9638763c73f7b626c1097"}
Jul 15 20:10:31.513366 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.513316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:31.614054 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:31.614023    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:32.423058 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:32.422999    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k" event={"ID":"80ad1e76-2889-4a8a-9654-2f815582ac3e","Type":"ContainerStarted","Data":"6232d12c5ed11d767e3f54b1cc9df7c64d057249e430e8a2f3ee435a06bd4d70"}
Jul 15 20:10:32.423511 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:32.423121    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:32.441432 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:32.441385    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k" podStartSLOduration=1.8985169750000002 podStartE2EDuration="2.44136929s" totalImagesPullingTime="542.852315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:10:32.437656954 +0000 UTC m=+2522.515449756" watchObservedRunningTime="2026-07-15 20:10:32.44136929 +0000 UTC m=+2522.519162091"
Jul 15 20:10:35.306875 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:35.306811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:38.434505 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:38.434479    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:10:44.162028 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:44.161974    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:44.262462 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:44.262409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:52.105296 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:52.105256    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:52.270984 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:52.270948    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:10:52.368158 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:10:52.368066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:11:17.960585 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:11:17.960554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:12:07.936792 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:12:07.936756    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:12:09.420555 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:12:09.420514    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:22.635690 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:22.635590    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:24.066048 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:24.066012    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:27.236198 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.236165    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"]
Jul 15 20:13:27.236605 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.236591    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy"
Jul 15 20:13:27.236652 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.236607    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy"
Jul 15 20:13:27.236691 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.236672    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7881ac1b-8b45-421a-8228-0fe03ed10c2a" containerName="istio-proxy"
Jul 15 20:13:27.239790 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.239769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.242554 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.242532    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-826a0f\"/\"sub-test-model-826a0f-kserve-self-signed-certs\""
Jul 15 20:13:27.242554 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.242542    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-826a0f\"/\"openshift-service-ca.crt\""
Jul 15 20:13:27.242554 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.242556    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-826a0f\"/\"default-dockercfg-ccrm8\""
Jul 15 20:13:27.242765 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.242550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-826a0f\"/\"kube-root-ca.crt\""
Jul 15 20:13:27.249926 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.249903    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"]
Jul 15 20:13:27.262120 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.262096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:27.264077 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.264057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-model-cache\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.264173 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.264111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tls-certs\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.264272 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.264195    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tmp-dir\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.264341 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.264278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qjn\" (UniqueName: \"kubernetes.io/projected/ff4ee9b7-a7b7-4693-a482-a9346caa9144-kube-api-access-z5qjn\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.264391 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.264346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-home\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.264443 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.264404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-dshm\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.365632 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.365588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-dshm\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.365815 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.365655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-model-cache\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.365815 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.365683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tls-certs\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.365815 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.365710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tmp-dir\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.365815 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.365728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qjn\" (UniqueName: \"kubernetes.io/projected/ff4ee9b7-a7b7-4693-a482-a9346caa9144-kube-api-access-z5qjn\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.365815 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.365762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-home\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.366114 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.366084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tmp-dir\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.366263 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.366145    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-model-cache\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.366263 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.366181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-home\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.368019 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.367998    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-dshm\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.368364 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.368347    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tls-certs\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.375626 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.375604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qjn\" (UniqueName: \"kubernetes.io/projected/ff4ee9b7-a7b7-4693-a482-a9346caa9144-kube-api-access-z5qjn\") pod \"sub-test-model-826a0f-kserve-8dd94f7d5-b45z7\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") " pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.550394 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.550300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:27.685501 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:27.685470    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"]
Jul 15 20:13:27.686360 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:13:27.686334    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff4ee9b7_a7b7_4693_a482_a9346caa9144.slice/crio-e062039910958bd077f34cdd53a36d5bf0278b3cdc5c21ca8245c75fe649b33f WatchSource:0}: Error finding container e062039910958bd077f34cdd53a36d5bf0278b3cdc5c21ca8245c75fe649b33f: Status 404 returned error can't find the container with id e062039910958bd077f34cdd53a36d5bf0278b3cdc5c21ca8245c75fe649b33f
Jul 15 20:13:28.466887 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:28.466853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7" event={"ID":"ff4ee9b7-a7b7-4693-a482-a9346caa9144","Type":"ContainerStarted","Data":"8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff"}
Jul 15 20:13:28.467346 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:28.466894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7" event={"ID":"ff4ee9b7-a7b7-4693-a482-a9346caa9144","Type":"ContainerStarted","Data":"e062039910958bd077f34cdd53a36d5bf0278b3cdc5c21ca8245c75fe649b33f"}
Jul 15 20:13:28.483739 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:28.483688    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7" podStartSLOduration=1.48367428 podStartE2EDuration="1.48367428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:13:28.481051849 +0000 UTC m=+2698.558844650" watchObservedRunningTime="2026-07-15 20:13:28.48367428 +0000 UTC m=+2698.561467081"
Jul 15 20:13:30.707154 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:30.707119    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 20:13:30.715353 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:30.715331    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlqrq_06b65950-8800-4fa8-bfdd-47574402da0d/ovn-acl-logging/0.log"
Jul 15 20:13:35.032514 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:35.032475    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:35.079632 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:35.079604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:37.551357 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:37.551322    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:37.551733 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:37.551369    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:37.564105 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:37.564083    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:38.542288 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:38.542261    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:13:45.145763 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:45.145725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:51.873335 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:51.873294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:13:51.924866 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:13:51.924837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:05.888722 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:05.888680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:05.994494 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:05.994440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:16.342087 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:16.342050    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"]
Jul 15 20:14:16.360392 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:16.360355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:17.347894 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:17.347853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:31.358787 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.358746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:31.363385 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.363355    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7" podUID="ff4ee9b7-a7b7-4693-a482-a9346caa9144" containerName="main" containerID="cri-o://8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff" gracePeriod=45
Jul 15 20:14:31.624273 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.624179    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:14:31.760679 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.760646    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tmp-dir\") pod \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") "
Jul 15 20:14:31.760679 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.760686    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tls-certs\") pod \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") "
Jul 15 20:14:31.760942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.760716    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5qjn\" (UniqueName: \"kubernetes.io/projected/ff4ee9b7-a7b7-4693-a482-a9346caa9144-kube-api-access-z5qjn\") pod \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") "
Jul 15 20:14:31.760942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.760760    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-dshm\") pod \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") "
Jul 15 20:14:31.761066 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.760951    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-home\") pod \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") "
Jul 15 20:14:31.761066 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.760967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff4ee9b7-a7b7-4693-a482-a9346caa9144" (UID: "ff4ee9b7-a7b7-4693-a482-a9346caa9144"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:14:31.761066 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.761008    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-model-cache\") pod \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\" (UID: \"ff4ee9b7-a7b7-4693-a482-a9346caa9144\") "
Jul 15 20:14:31.761262 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.761201    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-home" (OuterVolumeSpecName: "home") pod "ff4ee9b7-a7b7-4693-a482-a9346caa9144" (UID: "ff4ee9b7-a7b7-4693-a482-a9346caa9144"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:14:31.761339 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.761316    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff4ee9b7-a7b7-4693-a482-a9346caa9144" (UID: "ff4ee9b7-a7b7-4693-a482-a9346caa9144"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:14:31.761400 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.761342    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:14:31.761400 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.761358    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:14:31.763112 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.763089    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-dshm" (OuterVolumeSpecName: "dshm") pod "ff4ee9b7-a7b7-4693-a482-a9346caa9144" (UID: "ff4ee9b7-a7b7-4693-a482-a9346caa9144"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:14:31.763233 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.763138    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff4ee9b7-a7b7-4693-a482-a9346caa9144-kube-api-access-z5qjn" (OuterVolumeSpecName: "kube-api-access-z5qjn") pod "ff4ee9b7-a7b7-4693-a482-a9346caa9144" (UID: "ff4ee9b7-a7b7-4693-a482-a9346caa9144"). InnerVolumeSpecName "kube-api-access-z5qjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:14:31.763233 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.763146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff4ee9b7-a7b7-4693-a482-a9346caa9144" (UID: "ff4ee9b7-a7b7-4693-a482-a9346caa9144"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:14:31.858098 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.858062    2564 generic.go:356] "Generic (PLEG): container finished" podID="ff4ee9b7-a7b7-4693-a482-a9346caa9144" containerID="8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff" exitCode=0
Jul 15 20:14:31.858267 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.858110    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"
Jul 15 20:14:31.858267 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.858120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7" event={"ID":"ff4ee9b7-a7b7-4693-a482-a9346caa9144","Type":"ContainerDied","Data":"8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff"}
Jul 15 20:14:31.858267 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.858159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7" event={"ID":"ff4ee9b7-a7b7-4693-a482-a9346caa9144","Type":"ContainerDied","Data":"e062039910958bd077f34cdd53a36d5bf0278b3cdc5c21ca8245c75fe649b33f"}
Jul 15 20:14:31.858267 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.858177    2564 scope.go:117] "RemoveContainer" containerID="8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff"
Jul 15 20:14:31.862022 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.861996    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:14:31.862154 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.862027    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4ee9b7-a7b7-4693-a482-a9346caa9144-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:14:31.862154 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.862042    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5qjn\" (UniqueName: \"kubernetes.io/projected/ff4ee9b7-a7b7-4693-a482-a9346caa9144-kube-api-access-z5qjn\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:14:31.862154 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.862057    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4ee9b7-a7b7-4693-a482-a9346caa9144-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:14:31.868702 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.868687    2564 scope.go:117] "RemoveContainer" containerID="8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff"
Jul 15 20:14:31.868968 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:14:31.868947    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff\": container with ID starting with 8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff not found: ID does not exist" containerID="8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff"
Jul 15 20:14:31.869019 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.868976    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff"} err="failed to get container status \"8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff\": rpc error: code = NotFound desc = could not find container \"8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff\": container with ID starting with 8022cf0bbb767bc64936d5e15cf6aea1147d0f7fc21d0340814dbf05e5b0acff not found: ID does not exist"
Jul 15 20:14:31.879449 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.879393    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"]
Jul 15 20:14:31.885826 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:31.885799    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-826a0f/sub-test-model-826a0f-kserve-8dd94f7d5-b45z7"]
Jul 15 20:14:32.493815 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:32.493781    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff4ee9b7-a7b7-4693-a482-a9346caa9144" path="/var/lib/kubelet/pods/ff4ee9b7-a7b7-4693-a482-a9346caa9144/volumes"
Jul 15 20:14:39.484060 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:39.484023    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:14:52.380526 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:14:52.380437    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:00.135553 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.135517    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"]
Jul 15 20:15:00.136181 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.136154    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff4ee9b7-a7b7-4693-a482-a9346caa9144" containerName="main"
Jul 15 20:15:00.136181 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.136179    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4ee9b7-a7b7-4693-a482-a9346caa9144" containerName="main"
Jul 15 20:15:00.136427 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.136325    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff4ee9b7-a7b7-4693-a482-a9346caa9144" containerName="main"
Jul 15 20:15:00.139936 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.139917    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"]
Jul 15 20:15:00.140048 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.140028    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"
Jul 15 20:15:00.145250 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.142647    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:15:00.155844 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.155819    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"]
Jul 15 20:15:00.157390 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.155945    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"
Jul 15 20:15:00.162106 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.162067    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"]
Jul 15 20:15:00.162258 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.162111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"]
Jul 15 20:15:00.162258 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.162078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"
Jul 15 20:15:00.162258 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.162144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"
Jul 15 20:15:00.169681 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.169656    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"]
Jul 15 20:15:00.175753 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.175732    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"
Jul 15 20:15:00.213400 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.213374    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"
Jul 15 20:15:00.328052 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.328021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"]
Jul 15 20:15:00.329413 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:00.329388    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84c7e17_4971_4089_a67b_6e4ef2f2ec10.slice/crio-e91ab47f7c7b070d5906034d06fafe81ab45122b46c5eee3b7c67220d6cb2890 WatchSource:0}: Error finding container e91ab47f7c7b070d5906034d06fafe81ab45122b46c5eee3b7c67220d6cb2890: Status 404 returned error can't find the container with id e91ab47f7c7b070d5906034d06fafe81ab45122b46c5eee3b7c67220d6cb2890
Jul 15 20:15:00.360246 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.360198    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"]
Jul 15 20:15:00.370605 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:00.370574    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6157ccf2_360a_40c5_a80d_d0c22eb040c6.slice/crio-969e6247c904e6d079d95f127e4c278bec5756fb39d58c37ade04a1060b045d1 WatchSource:0}: Error finding container 969e6247c904e6d079d95f127e4c278bec5756fb39d58c37ade04a1060b045d1: Status 404 returned error can't find the container with id 969e6247c904e6d079d95f127e4c278bec5756fb39d58c37ade04a1060b045d1
Jul 15 20:15:00.389047 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:00.389025    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"]
Jul 15 20:15:00.390932 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:00.390903    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f99189_81dc_4cb9_90af_deade6fec341.slice/crio-d19acf4a5a8a7beea38c671d5948b58cf12ff7786e774bc8381da06930905054 WatchSource:0}: Error finding container d19acf4a5a8a7beea38c671d5948b58cf12ff7786e774bc8381da06930905054: Status 404 returned error can't find the container with id d19acf4a5a8a7beea38c671d5948b58cf12ff7786e774bc8381da06930905054
Jul 15 20:15:01.026949 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.026912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq" event={"ID":"c3f99189-81dc-4cb9-90af-deade6fec341","Type":"ContainerStarted","Data":"bc7b260eb9212acbe524bec91e4445fed507d294320223da8b21001409a0f290"}
Jul 15 20:15:01.027149 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.026971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq" event={"ID":"c3f99189-81dc-4cb9-90af-deade6fec341","Type":"ContainerStarted","Data":"d19acf4a5a8a7beea38c671d5948b58cf12ff7786e774bc8381da06930905054"}
Jul 15 20:15:01.028522 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.028494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q" event={"ID":"c84c7e17-4971-4089-a67b-6e4ef2f2ec10","Type":"ContainerStarted","Data":"d7ea8a024622e83b9f7270cb9691e302119fde85b1d7a9923fbb6746c2ed5668"}
Jul 15 20:15:01.028632 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.028542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q" event={"ID":"c84c7e17-4971-4089-a67b-6e4ef2f2ec10","Type":"ContainerStarted","Data":"e91ab47f7c7b070d5906034d06fafe81ab45122b46c5eee3b7c67220d6cb2890"}
Jul 15 20:15:01.029931 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.029904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d" event={"ID":"6157ccf2-360a-40c5-a80d-d0c22eb040c6","Type":"ContainerStarted","Data":"5fcb447f96361f24519a3419383206bf5714f2e08a380be4b422a383d16dcb7d"}
Jul 15 20:15:01.030019 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.029948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d" event={"ID":"6157ccf2-360a-40c5-a80d-d0c22eb040c6","Type":"ContainerStarted","Data":"969e6247c904e6d079d95f127e4c278bec5756fb39d58c37ade04a1060b045d1"}
Jul 15 20:15:01.042878 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.042830    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq" podStartSLOduration=1.042815884 podStartE2EDuration="1.042815884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:15:01.039739928 +0000 UTC m=+2791.117532723" watchObservedRunningTime="2026-07-15 20:15:01.042815884 +0000 UTC m=+2791.120608685"
Jul 15 20:15:01.055022 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.054979    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d" podStartSLOduration=1.054965803 podStartE2EDuration="1.054965803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:15:01.051979454 +0000 UTC m=+2791.129772255" watchObservedRunningTime="2026-07-15 20:15:01.054965803 +0000 UTC m=+2791.132758604"
Jul 15 20:15:01.064473 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:01.064392    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q" podStartSLOduration=1.064376015 podStartE2EDuration="1.064376015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:15:01.063964591 +0000 UTC m=+2791.141757397" watchObservedRunningTime="2026-07-15 20:15:01.064376015 +0000 UTC m=+2791.142168817"
Jul 15 20:15:02.037436 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:02.037401    2564 generic.go:356] "Generic (PLEG): container finished" podID="c84c7e17-4971-4089-a67b-6e4ef2f2ec10" containerID="d7ea8a024622e83b9f7270cb9691e302119fde85b1d7a9923fbb6746c2ed5668" exitCode=0
Jul 15 20:15:02.037883 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:02.037491    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q" event={"ID":"c84c7e17-4971-4089-a67b-6e4ef2f2ec10","Type":"ContainerDied","Data":"d7ea8a024622e83b9f7270cb9691e302119fde85b1d7a9923fbb6746c2ed5668"}
Jul 15 20:15:02.039064 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:02.039042    2564 generic.go:356] "Generic (PLEG): container finished" podID="6157ccf2-360a-40c5-a80d-d0c22eb040c6" containerID="5fcb447f96361f24519a3419383206bf5714f2e08a380be4b422a383d16dcb7d" exitCode=0
Jul 15 20:15:02.039140 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:02.039120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d" event={"ID":"6157ccf2-360a-40c5-a80d-d0c22eb040c6","Type":"ContainerDied","Data":"5fcb447f96361f24519a3419383206bf5714f2e08a380be4b422a383d16dcb7d"}
Jul 15 20:15:02.040647 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:02.040622    2564 generic.go:356] "Generic (PLEG): container finished" podID="c3f99189-81dc-4cb9-90af-deade6fec341" containerID="bc7b260eb9212acbe524bec91e4445fed507d294320223da8b21001409a0f290" exitCode=0
Jul 15 20:15:02.040774 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:02.040649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq" event={"ID":"c3f99189-81dc-4cb9-90af-deade6fec341","Type":"ContainerDied","Data":"bc7b260eb9212acbe524bec91e4445fed507d294320223da8b21001409a0f290"}
Jul 15 20:15:03.214596 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:03.214566    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"
Jul 15 20:15:03.268838 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:03.268814    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"
Jul 15 20:15:03.273074 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:03.273053    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"
Jul 15 20:15:04.054997 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.054968    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"
Jul 15 20:15:04.055212 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.054998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q" event={"ID":"c84c7e17-4971-4089-a67b-6e4ef2f2ec10","Type":"ContainerDied","Data":"e91ab47f7c7b070d5906034d06fafe81ab45122b46c5eee3b7c67220d6cb2890"}
Jul 15 20:15:04.055212 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.055050    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e91ab47f7c7b070d5906034d06fafe81ab45122b46c5eee3b7c67220d6cb2890"
Jul 15 20:15:04.056402 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.056387    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d"
Jul 15 20:15:04.056490 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.056430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-6k47d" event={"ID":"6157ccf2-360a-40c5-a80d-d0c22eb040c6","Type":"ContainerDied","Data":"969e6247c904e6d079d95f127e4c278bec5756fb39d58c37ade04a1060b045d1"}
Jul 15 20:15:04.056490 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.056473    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="969e6247c904e6d079d95f127e4c278bec5756fb39d58c37ade04a1060b045d1"
Jul 15 20:15:04.057822 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.057792    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"
Jul 15 20:15:04.057822 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.057802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq" event={"ID":"c3f99189-81dc-4cb9-90af-deade6fec341","Type":"ContainerDied","Data":"d19acf4a5a8a7beea38c671d5948b58cf12ff7786e774bc8381da06930905054"}
Jul 15 20:15:04.057939 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:04.057841    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d19acf4a5a8a7beea38c671d5948b58cf12ff7786e774bc8381da06930905054"
Jul 15 20:15:30.315637 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:30.315588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:36.939763 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:36.939730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:37.841807 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:37.841770    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:38.855943 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.855900    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-139858168905392"]
Jul 15 20:15:38.856527 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856505    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f99189-81dc-4cb9-90af-deade6fec341" containerName="cleanup"
Jul 15 20:15:38.856618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856530    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f99189-81dc-4cb9-90af-deade6fec341" containerName="cleanup"
Jul 15 20:15:38.856618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856544    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6157ccf2-360a-40c5-a80d-d0c22eb040c6" containerName="cleanup"
Jul 15 20:15:38.856618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856552    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6157ccf2-360a-40c5-a80d-d0c22eb040c6" containerName="cleanup"
Jul 15 20:15:38.856618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856570    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84c7e17-4971-4089-a67b-6e4ef2f2ec10" containerName="cleanup"
Jul 15 20:15:38.856618 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856578    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84c7e17-4971-4089-a67b-6e4ef2f2ec10" containerName="cleanup"
Jul 15 20:15:38.856854 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856681    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f99189-81dc-4cb9-90af-deade6fec341" containerName="cleanup"
Jul 15 20:15:38.856854 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856696    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6157ccf2-360a-40c5-a80d-d0c22eb040c6" containerName="cleanup"
Jul 15 20:15:38.856854 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.856710    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c84c7e17-4971-4089-a67b-6e4ef2f2ec10" containerName="cleanup"
Jul 15 20:15:38.859937 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.859915    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:38.862025 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.862008    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6p672\""
Jul 15 20:15:38.866366 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:38.866343    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168905392"]
Jul 15 20:15:39.002658 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:39.002609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsvcc\" (UniqueName: \"kubernetes.io/projected/c354fb96-2368-4dd4-b086-fc76d5f87c2e-kube-api-access-jsvcc\") pod \"test-curl-7613-139858168905392\" (UID: \"c354fb96-2368-4dd4-b086-fc76d5f87c2e\") " pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:39.103587 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:39.103543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsvcc\" (UniqueName: \"kubernetes.io/projected/c354fb96-2368-4dd4-b086-fc76d5f87c2e-kube-api-access-jsvcc\") pod \"test-curl-7613-139858168905392\" (UID: \"c354fb96-2368-4dd4-b086-fc76d5f87c2e\") " pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:39.111925 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:39.111864    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsvcc\" (UniqueName: \"kubernetes.io/projected/c354fb96-2368-4dd4-b086-fc76d5f87c2e-kube-api-access-jsvcc\") pod \"test-curl-7613-139858168905392\" (UID: \"c354fb96-2368-4dd4-b086-fc76d5f87c2e\") " pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:39.169515 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:39.169485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:39.310018 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:39.309990    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168905392"]
Jul 15 20:15:39.310999 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:39.310974    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc354fb96_2368_4dd4_b086_fc76d5f87c2e.slice/crio-8819a5a4b4a24ee0cce6c986c018943df9895a642e9738ed5653cc7358bad040 WatchSource:0}: Error finding container 8819a5a4b4a24ee0cce6c986c018943df9895a642e9738ed5653cc7358bad040: Status 404 returned error can't find the container with id 8819a5a4b4a24ee0cce6c986c018943df9895a642e9738ed5653cc7358bad040
Jul 15 20:15:39.312338 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:39.312320    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:15:40.275932 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:40.275844    2564 generic.go:356] "Generic (PLEG): container finished" podID="c354fb96-2368-4dd4-b086-fc76d5f87c2e" containerID="d260a0f328899680c449eb7cc70a7111b0f8dadcea5ccb05bb1cc046f33bb462" exitCode=0
Jul 15 20:15:40.275932 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:40.275923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168905392" event={"ID":"c354fb96-2368-4dd4-b086-fc76d5f87c2e","Type":"ContainerDied","Data":"d260a0f328899680c449eb7cc70a7111b0f8dadcea5ccb05bb1cc046f33bb462"}
Jul 15 20:15:40.276339 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:40.275962    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168905392" event={"ID":"c354fb96-2368-4dd4-b086-fc76d5f87c2e","Type":"ContainerStarted","Data":"8819a5a4b4a24ee0cce6c986c018943df9895a642e9738ed5653cc7358bad040"}
Jul 15 20:15:41.417236 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.417201    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:41.425861 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.425836    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsvcc\" (UniqueName: \"kubernetes.io/projected/c354fb96-2368-4dd4-b086-fc76d5f87c2e-kube-api-access-jsvcc\") pod \"c354fb96-2368-4dd4-b086-fc76d5f87c2e\" (UID: \"c354fb96-2368-4dd4-b086-fc76d5f87c2e\") "
Jul 15 20:15:41.427957 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.427926    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c354fb96-2368-4dd4-b086-fc76d5f87c2e-kube-api-access-jsvcc" (OuterVolumeSpecName: "kube-api-access-jsvcc") pod "c354fb96-2368-4dd4-b086-fc76d5f87c2e" (UID: "c354fb96-2368-4dd4-b086-fc76d5f87c2e"). InnerVolumeSpecName "kube-api-access-jsvcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:15:41.436036 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.436008    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7613-139858168905392_c354fb96-2368-4dd4-b086-fc76d5f87c2e/test-curl-7613-139858168905392/0.log"
Jul 15 20:15:41.461461 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.461426    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-139858168905392"]
Jul 15 20:15:41.464980 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.464956    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-139858168905392"]
Jul 15 20:15:41.527450 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.527418    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsvcc\" (UniqueName: \"kubernetes.io/projected/c354fb96-2368-4dd4-b086-fc76d5f87c2e-kube-api-access-jsvcc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:15:41.585246 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.585136    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-139858168905008"]
Jul 15 20:15:41.585661 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.585639    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c354fb96-2368-4dd4-b086-fc76d5f87c2e" containerName="test-curl-7613-139858168905392"
Jul 15 20:15:41.585765 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.585664    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c354fb96-2368-4dd4-b086-fc76d5f87c2e" containerName="test-curl-7613-139858168905392"
Jul 15 20:15:41.585821 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.585774    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c354fb96-2368-4dd4-b086-fc76d5f87c2e" containerName="test-curl-7613-139858168905392"
Jul 15 20:15:41.588904 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.588883    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:41.593838 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.593816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168905008"]
Jul 15 20:15:41.628369 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.628336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt8lj\" (UniqueName: \"kubernetes.io/projected/e8ae19d0-7a22-4412-8551-d79a2c346945-kube-api-access-rt8lj\") pod \"test-curl-7613-139858168905008\" (UID: \"e8ae19d0-7a22-4412-8551-d79a2c346945\") " pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:41.728959 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.728915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt8lj\" (UniqueName: \"kubernetes.io/projected/e8ae19d0-7a22-4412-8551-d79a2c346945-kube-api-access-rt8lj\") pod \"test-curl-7613-139858168905008\" (UID: \"e8ae19d0-7a22-4412-8551-d79a2c346945\") " pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:41.737019 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.736991    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt8lj\" (UniqueName: \"kubernetes.io/projected/e8ae19d0-7a22-4412-8551-d79a2c346945-kube-api-access-rt8lj\") pod \"test-curl-7613-139858168905008\" (UID: \"e8ae19d0-7a22-4412-8551-d79a2c346945\") " pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:41.899664 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:41.899575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:42.041800 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:42.041774    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168905008"]
Jul 15 20:15:42.042476 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:42.042450    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8ae19d0_7a22_4412_8551_d79a2c346945.slice/crio-c0cd4280de85882f074c330375f9130f39c28c4b9a09fb1964c4180f0c5d8a89 WatchSource:0}: Error finding container c0cd4280de85882f074c330375f9130f39c28c4b9a09fb1964c4180f0c5d8a89: Status 404 returned error can't find the container with id c0cd4280de85882f074c330375f9130f39c28c4b9a09fb1964c4180f0c5d8a89
Jul 15 20:15:42.289678 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:42.289644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168905008" event={"ID":"e8ae19d0-7a22-4412-8551-d79a2c346945","Type":"ContainerStarted","Data":"c0cd4280de85882f074c330375f9130f39c28c4b9a09fb1964c4180f0c5d8a89"}
Jul 15 20:15:42.290896 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:42.290876    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:42.291020 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:42.290898    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8819a5a4b4a24ee0cce6c986c018943df9895a642e9738ed5653cc7358bad040"
Jul 15 20:15:42.301421 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:42.301394    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168905392\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="c354fb96-2368-4dd4-b086-fc76d5f87c2e" pod="opendatahub/test-curl-7613-139858168905392"
Jul 15 20:15:42.493566 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:42.493530    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c354fb96-2368-4dd4-b086-fc76d5f87c2e" path="/var/lib/kubelet/pods/c354fb96-2368-4dd4-b086-fc76d5f87c2e/volumes"
Jul 15 20:15:43.298897 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:43.298865    2564 generic.go:356] "Generic (PLEG): container finished" podID="e8ae19d0-7a22-4412-8551-d79a2c346945" containerID="192afa4ef50c92640306470b252ae4c0a247457125d79d02770c9cc737cd6c71" exitCode=0
Jul 15 20:15:43.299075 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:43.298955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168905008" event={"ID":"e8ae19d0-7a22-4412-8551-d79a2c346945","Type":"ContainerDied","Data":"192afa4ef50c92640306470b252ae4c0a247457125d79d02770c9cc737cd6c71"}
Jul 15 20:15:44.443822 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.443793    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:44.455868 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.455835    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt8lj\" (UniqueName: \"kubernetes.io/projected/e8ae19d0-7a22-4412-8551-d79a2c346945-kube-api-access-rt8lj\") pod \"e8ae19d0-7a22-4412-8551-d79a2c346945\" (UID: \"e8ae19d0-7a22-4412-8551-d79a2c346945\") "
Jul 15 20:15:44.465176 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.464780    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7613-139858168905008_e8ae19d0-7a22-4412-8551-d79a2c346945/test-curl-7613-139858168905008/0.log"
Jul 15 20:15:44.469441 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.468562    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8ae19d0-7a22-4412-8551-d79a2c346945-kube-api-access-rt8lj" (OuterVolumeSpecName: "kube-api-access-rt8lj") pod "e8ae19d0-7a22-4412-8551-d79a2c346945" (UID: "e8ae19d0-7a22-4412-8551-d79a2c346945"). InnerVolumeSpecName "kube-api-access-rt8lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:15:44.486332 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.486300    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-139858168905008"]
Jul 15 20:15:44.495865 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.495841    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-139858168905008"]
Jul 15 20:15:44.556703 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.556619    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt8lj\" (UniqueName: \"kubernetes.io/projected/e8ae19d0-7a22-4412-8551-d79a2c346945-kube-api-access-rt8lj\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:15:44.653403 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.653368    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:44.653816 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.653800    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ae19d0-7a22-4412-8551-d79a2c346945" containerName="test-curl-7613-139858168905008"
Jul 15 20:15:44.653869 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.653821    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ae19d0-7a22-4412-8551-d79a2c346945" containerName="test-curl-7613-139858168905008"
Jul 15 20:15:44.653907 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.653897    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ae19d0-7a22-4412-8551-d79a2c346945" containerName="test-curl-7613-139858168905008"
Jul 15 20:15:44.657062 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.657036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:44.657319 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.657299    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-254ch\" (UniqueName: \"kubernetes.io/projected/0c1791cb-8ded-41c2-8878-e4dbd792e100-kube-api-access-254ch\") pod \"test-curl-7613-139858168841536\" (UID: \"0c1791cb-8ded-41c2-8878-e4dbd792e100\") " pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:44.663078 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.663054    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:44.758167 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.758128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-254ch\" (UniqueName: \"kubernetes.io/projected/0c1791cb-8ded-41c2-8878-e4dbd792e100-kube-api-access-254ch\") pod \"test-curl-7613-139858168841536\" (UID: \"0c1791cb-8ded-41c2-8878-e4dbd792e100\") " pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:44.766363 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.766339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-254ch\" (UniqueName: \"kubernetes.io/projected/0c1791cb-8ded-41c2-8878-e4dbd792e100-kube-api-access-254ch\") pod \"test-curl-7613-139858168841536\" (UID: \"0c1791cb-8ded-41c2-8878-e4dbd792e100\") " pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:44.902032 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.901943    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:44.967472 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:44.967442    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:45.004836 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:45.004801    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:45.103144 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:45.103116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:45.104239 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:45.104194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c1791cb_8ded_41c2_8878_e4dbd792e100.slice/crio-d4a12204d813f8c77c843b2f1d7c39d4f9ec67f2def0a6300035a79b525a83d8 WatchSource:0}: Error finding container d4a12204d813f8c77c843b2f1d7c39d4f9ec67f2def0a6300035a79b525a83d8: Status 404 returned error can't find the container with id d4a12204d813f8c77c843b2f1d7c39d4f9ec67f2def0a6300035a79b525a83d8
Jul 15 20:15:45.311871 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:45.311833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168841536" event={"ID":"0c1791cb-8ded-41c2-8878-e4dbd792e100","Type":"ContainerStarted","Data":"d4a12204d813f8c77c843b2f1d7c39d4f9ec67f2def0a6300035a79b525a83d8"}
Jul 15 20:15:45.313255 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:45.313203    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0cd4280de85882f074c330375f9130f39c28c4b9a09fb1964c4180f0c5d8a89"
Jul 15 20:15:45.313382 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:45.313236    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:45.317884 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:45.317854    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168905008\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e8ae19d0-7a22-4412-8551-d79a2c346945" pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:46.321598 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:46.321561    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c1791cb-8ded-41c2-8878-e4dbd792e100" containerID="07523178012f48be432ab8d055d13a358e9319a7cfbbadf9277415868d7d2981" exitCode=0
Jul 15 20:15:46.321983 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:46.321638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168841536" event={"ID":"0c1791cb-8ded-41c2-8878-e4dbd792e100","Type":"ContainerDied","Data":"07523178012f48be432ab8d055d13a358e9319a7cfbbadf9277415868d7d2981"}
Jul 15 20:15:46.323507 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:46.323483    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168905008\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e8ae19d0-7a22-4412-8551-d79a2c346945" pod="opendatahub/test-curl-7613-139858168905008"
Jul 15 20:15:46.493311 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:46.493277    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ae19d0-7a22-4412-8551-d79a2c346945" path="/var/lib/kubelet/pods/e8ae19d0-7a22-4412-8551-d79a2c346945/volumes"
Jul 15 20:15:47.466775 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.466753    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:47.480342 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.480003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-254ch\" (UniqueName: \"kubernetes.io/projected/0c1791cb-8ded-41c2-8878-e4dbd792e100-kube-api-access-254ch\") pod \"0c1791cb-8ded-41c2-8878-e4dbd792e100\" (UID: \"0c1791cb-8ded-41c2-8878-e4dbd792e100\") "
Jul 15 20:15:47.482649 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.482620    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1791cb-8ded-41c2-8878-e4dbd792e100-kube-api-access-254ch" (OuterVolumeSpecName: "kube-api-access-254ch") pod "0c1791cb-8ded-41c2-8878-e4dbd792e100" (UID: "0c1791cb-8ded-41c2-8878-e4dbd792e100"). InnerVolumeSpecName "kube-api-access-254ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:15:47.485832 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.485810    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7613-139858168841536_0c1791cb-8ded-41c2-8878-e4dbd792e100/test-curl-7613-139858168841536/0.log"
Jul 15 20:15:47.510310 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.510276    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:47.512906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.512883    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:47.582648 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.582559    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-254ch\" (UniqueName: \"kubernetes.io/projected/0c1791cb-8ded-41c2-8878-e4dbd792e100-kube-api-access-254ch\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:15:47.630718 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.630682    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:47.631094 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.631080    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1791cb-8ded-41c2-8878-e4dbd792e100" containerName="test-curl-7613-139858168841536"
Jul 15 20:15:47.631148 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.631095    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1791cb-8ded-41c2-8878-e4dbd792e100" containerName="test-curl-7613-139858168841536"
Jul 15 20:15:47.631263 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.631172    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1791cb-8ded-41c2-8878-e4dbd792e100" containerName="test-curl-7613-139858168841536"
Jul 15 20:15:47.634356 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.634339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:47.639696 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.639660    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:47.683306 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.683280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hz5\" (UniqueName: \"kubernetes.io/projected/1b25058e-a330-4bf1-babf-61d2e6369a11-kube-api-access-w6hz5\") pod \"test-curl-7613-139858168841536\" (UID: \"1b25058e-a330-4bf1-babf-61d2e6369a11\") " pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:47.783913 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.783880    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hz5\" (UniqueName: \"kubernetes.io/projected/1b25058e-a330-4bf1-babf-61d2e6369a11-kube-api-access-w6hz5\") pod \"test-curl-7613-139858168841536\" (UID: \"1b25058e-a330-4bf1-babf-61d2e6369a11\") " pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:47.791352 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.791326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hz5\" (UniqueName: \"kubernetes.io/projected/1b25058e-a330-4bf1-babf-61d2e6369a11-kube-api-access-w6hz5\") pod \"test-curl-7613-139858168841536\" (UID: \"1b25058e-a330-4bf1-babf-61d2e6369a11\") " pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:47.945643 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:47.945554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:48.077168 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:48.077144    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b25058e_a330_4bf1_babf_61d2e6369a11.slice/crio-4ffa30f51924bfc335037d8ddea7b2d6d3f60607c2651593c3b1a5ba4b6a7d7e WatchSource:0}: Error finding container 4ffa30f51924bfc335037d8ddea7b2d6d3f60607c2651593c3b1a5ba4b6a7d7e: Status 404 returned error can't find the container with id 4ffa30f51924bfc335037d8ddea7b2d6d3f60607c2651593c3b1a5ba4b6a7d7e
Jul 15 20:15:48.081591 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:48.078258    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:48.334283 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:48.334244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168841536" event={"ID":"1b25058e-a330-4bf1-babf-61d2e6369a11","Type":"ContainerStarted","Data":"4ffa30f51924bfc335037d8ddea7b2d6d3f60607c2651593c3b1a5ba4b6a7d7e"}
Jul 15 20:15:48.335524 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:48.335506    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4a12204d813f8c77c843b2f1d7c39d4f9ec67f2def0a6300035a79b525a83d8"
Jul 15 20:15:48.335524 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:48.335521    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:48.346464 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:48.346427    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7613-139858168841536" oldPodUID="0c1791cb-8ded-41c2-8878-e4dbd792e100" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11"
Jul 15 20:15:48.493027 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:48.493000    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c1791cb-8ded-41c2-8878-e4dbd792e100" path="/var/lib/kubelet/pods/0c1791cb-8ded-41c2-8878-e4dbd792e100/volumes"
Jul 15 20:15:49.343190 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:49.343153    2564 generic.go:356] "Generic (PLEG): container finished" podID="1b25058e-a330-4bf1-babf-61d2e6369a11" containerID="041eaa9914a8dc42c2d70c21b84c114d6e00efb41deecfd6d67e1ac818b5b930" exitCode=0
Jul 15 20:15:49.343388 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:49.343250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168841536" event={"ID":"1b25058e-a330-4bf1-babf-61d2e6369a11","Type":"ContainerDied","Data":"041eaa9914a8dc42c2d70c21b84c114d6e00efb41deecfd6d67e1ac818b5b930"}
Jul 15 20:15:50.490735 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.490706    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:50.508031 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.508002    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6hz5\" (UniqueName: \"kubernetes.io/projected/1b25058e-a330-4bf1-babf-61d2e6369a11-kube-api-access-w6hz5\") pod \"1b25058e-a330-4bf1-babf-61d2e6369a11\" (UID: \"1b25058e-a330-4bf1-babf-61d2e6369a11\") "
Jul 15 20:15:50.508698 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.508410    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7613-139858168841536_1b25058e-a330-4bf1-babf-61d2e6369a11/test-curl-7613-139858168841536/0.log"
Jul 15 20:15:50.510674 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.510649    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b25058e-a330-4bf1-babf-61d2e6369a11-kube-api-access-w6hz5" (OuterVolumeSpecName: "kube-api-access-w6hz5") pod "1b25058e-a330-4bf1-babf-61d2e6369a11" (UID: "1b25058e-a330-4bf1-babf-61d2e6369a11"). InnerVolumeSpecName "kube-api-access-w6hz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:15:50.535051 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.535023    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:50.539067 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.539045    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-139858168841536"]
Jul 15 20:15:50.609884 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.609808    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6hz5\" (UniqueName: \"kubernetes.io/projected/1b25058e-a330-4bf1-babf-61d2e6369a11-kube-api-access-w6hz5\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:15:50.661235 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.661193    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:50.661597 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.661584    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11" containerName="test-curl-7613-139858168841536"
Jul 15 20:15:50.661637 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.661599    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11" containerName="test-curl-7613-139858168841536"
Jul 15 20:15:50.661708 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.661698    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11" containerName="test-curl-7613-139858168841536"
Jul 15 20:15:50.664717 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.664702    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:50.670022 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.670000    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:50.711163 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.711132    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj6h2\" (UniqueName: \"kubernetes.io/projected/14355176-82a2-4c30-a4a8-82f3cebe6307-kube-api-access-mj6h2\") pod \"test-curl-7613-139858168842544\" (UID: \"14355176-82a2-4c30-a4a8-82f3cebe6307\") " pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:50.812462 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.812433    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj6h2\" (UniqueName: \"kubernetes.io/projected/14355176-82a2-4c30-a4a8-82f3cebe6307-kube-api-access-mj6h2\") pod \"test-curl-7613-139858168842544\" (UID: \"14355176-82a2-4c30-a4a8-82f3cebe6307\") " pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:50.820082 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.820050    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj6h2\" (UniqueName: \"kubernetes.io/projected/14355176-82a2-4c30-a4a8-82f3cebe6307-kube-api-access-mj6h2\") pod \"test-curl-7613-139858168842544\" (UID: \"14355176-82a2-4c30-a4a8-82f3cebe6307\") " pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:50.975471 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:50.975439    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:51.109093 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:51.109065    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:51.110414 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:51.110380    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14355176_82a2_4c30_a4a8_82f3cebe6307.slice/crio-0a7e18f336302bb70d5a9fc1666fe511c3eefd529c87699d63ad46eb77ae69ef WatchSource:0}: Error finding container 0a7e18f336302bb70d5a9fc1666fe511c3eefd529c87699d63ad46eb77ae69ef: Status 404 returned error can't find the container with id 0a7e18f336302bb70d5a9fc1666fe511c3eefd529c87699d63ad46eb77ae69ef
Jul 15 20:15:51.356691 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:51.356607    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:51.356691 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:51.356621    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ffa30f51924bfc335037d8ddea7b2d6d3f60607c2651593c3b1a5ba4b6a7d7e"
Jul 15 20:15:51.357894 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:51.357872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168842544" event={"ID":"14355176-82a2-4c30-a4a8-82f3cebe6307","Type":"ContainerStarted","Data":"0a7e18f336302bb70d5a9fc1666fe511c3eefd529c87699d63ad46eb77ae69ef"}
Jul 15 20:15:51.366779 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:51.366753    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168841536\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:52.365684 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:52.365651    2564 generic.go:356] "Generic (PLEG): container finished" podID="14355176-82a2-4c30-a4a8-82f3cebe6307" containerID="9f7771642e903ef2f42407290ca1e912550fb33aabf85fdf2aadd08b0b1c5718" exitCode=0
Jul 15 20:15:52.366066 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:52.365773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168842544" event={"ID":"14355176-82a2-4c30-a4a8-82f3cebe6307","Type":"ContainerDied","Data":"9f7771642e903ef2f42407290ca1e912550fb33aabf85fdf2aadd08b0b1c5718"}
Jul 15 20:15:52.379868 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:52.379824    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168841536\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11" pod="opendatahub/test-curl-7613-139858168841536"
Jul 15 20:15:52.493042 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:52.493010    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b25058e-a330-4bf1-babf-61d2e6369a11" path="/var/lib/kubelet/pods/1b25058e-a330-4bf1-babf-61d2e6369a11/volumes"
Jul 15 20:15:53.025403 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.025369    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:53.118783 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.118746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:53.507720 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.507695    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:53.526398 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.526367    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7613-139858168842544_14355176-82a2-4c30-a4a8-82f3cebe6307/test-curl-7613-139858168842544/0.log"
Jul 15 20:15:53.537651 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.537627    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj6h2\" (UniqueName: \"kubernetes.io/projected/14355176-82a2-4c30-a4a8-82f3cebe6307-kube-api-access-mj6h2\") pod \"14355176-82a2-4c30-a4a8-82f3cebe6307\" (UID: \"14355176-82a2-4c30-a4a8-82f3cebe6307\") "
Jul 15 20:15:53.539755 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.539732    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14355176-82a2-4c30-a4a8-82f3cebe6307-kube-api-access-mj6h2" (OuterVolumeSpecName: "kube-api-access-mj6h2") pod "14355176-82a2-4c30-a4a8-82f3cebe6307" (UID: "14355176-82a2-4c30-a4a8-82f3cebe6307"). InnerVolumeSpecName "kube-api-access-mj6h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:15:53.565473 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.565446    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:53.577198 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.577176    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:53.638569 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.638497    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj6h2\" (UniqueName: \"kubernetes.io/projected/14355176-82a2-4c30-a4a8-82f3cebe6307-kube-api-access-mj6h2\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:15:53.710456 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.710418    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:53.710857 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.710844    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14355176-82a2-4c30-a4a8-82f3cebe6307" containerName="test-curl-7613-139858168842544"
Jul 15 20:15:53.710902 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.710859    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="14355176-82a2-4c30-a4a8-82f3cebe6307" containerName="test-curl-7613-139858168842544"
Jul 15 20:15:53.710933 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.710926    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="14355176-82a2-4c30-a4a8-82f3cebe6307" containerName="test-curl-7613-139858168842544"
Jul 15 20:15:53.714089 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.714043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:53.718296 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.718271    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:53.739787 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.739761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shg79\" (UniqueName: \"kubernetes.io/projected/6214b0a5-60b7-4255-82ec-25ee8bb5649d-kube-api-access-shg79\") pod \"test-curl-7613-139858168842544\" (UID: \"6214b0a5-60b7-4255-82ec-25ee8bb5649d\") " pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:53.841167 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.841125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shg79\" (UniqueName: \"kubernetes.io/projected/6214b0a5-60b7-4255-82ec-25ee8bb5649d-kube-api-access-shg79\") pod \"test-curl-7613-139858168842544\" (UID: \"6214b0a5-60b7-4255-82ec-25ee8bb5649d\") " pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:53.849234 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:53.849206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shg79\" (UniqueName: \"kubernetes.io/projected/6214b0a5-60b7-4255-82ec-25ee8bb5649d-kube-api-access-shg79\") pod \"test-curl-7613-139858168842544\" (UID: \"6214b0a5-60b7-4255-82ec-25ee8bb5649d\") " pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:54.023993 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.023940    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:54.161972 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.161944    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:54.163079 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:54.163055    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6214b0a5_60b7_4255_82ec_25ee8bb5649d.slice/crio-427d78cbf2ff8d7048c2421876fa2615996e9ac81efdc1eb21fdbb8bcf4fcb33 WatchSource:0}: Error finding container 427d78cbf2ff8d7048c2421876fa2615996e9ac81efdc1eb21fdbb8bcf4fcb33: Status 404 returned error can't find the container with id 427d78cbf2ff8d7048c2421876fa2615996e9ac81efdc1eb21fdbb8bcf4fcb33
Jul 15 20:15:54.379437 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.379348    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:54.379437 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.379403    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a7e18f336302bb70d5a9fc1666fe511c3eefd529c87699d63ad46eb77ae69ef"
Jul 15 20:15:54.380848 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.380820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168842544" event={"ID":"6214b0a5-60b7-4255-82ec-25ee8bb5649d","Type":"ContainerStarted","Data":"427d78cbf2ff8d7048c2421876fa2615996e9ac81efdc1eb21fdbb8bcf4fcb33"}
Jul 15 20:15:54.390265 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.390216    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7613-139858168842544" oldPodUID="14355176-82a2-4c30-a4a8-82f3cebe6307" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d"
Jul 15 20:15:54.494787 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:54.494753    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14355176-82a2-4c30-a4a8-82f3cebe6307" path="/var/lib/kubelet/pods/14355176-82a2-4c30-a4a8-82f3cebe6307/volumes"
Jul 15 20:15:55.387845 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:55.387815    2564 generic.go:356] "Generic (PLEG): container finished" podID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" containerID="875264677a9f82bcae2746030e26dc5f5ee2e35685976d10336fda0ede997320" exitCode=0
Jul 15 20:15:55.388266 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:55.387912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-139858168842544" event={"ID":"6214b0a5-60b7-4255-82ec-25ee8bb5649d","Type":"ContainerDied","Data":"875264677a9f82bcae2746030e26dc5f5ee2e35685976d10336fda0ede997320"}
Jul 15 20:15:56.540341 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.540315    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:56.558952 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.558927    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7613-139858168842544_6214b0a5-60b7-4255-82ec-25ee8bb5649d/test-curl-7613-139858168842544/0.log"
Jul 15 20:15:56.566076 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.566056    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shg79\" (UniqueName: \"kubernetes.io/projected/6214b0a5-60b7-4255-82ec-25ee8bb5649d-kube-api-access-shg79\") pod \"6214b0a5-60b7-4255-82ec-25ee8bb5649d\" (UID: \"6214b0a5-60b7-4255-82ec-25ee8bb5649d\") "
Jul 15 20:15:56.568390 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.568355    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6214b0a5-60b7-4255-82ec-25ee8bb5649d-kube-api-access-shg79" (OuterVolumeSpecName: "kube-api-access-shg79") pod "6214b0a5-60b7-4255-82ec-25ee8bb5649d" (UID: "6214b0a5-60b7-4255-82ec-25ee8bb5649d"). InnerVolumeSpecName "kube-api-access-shg79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:15:56.585020 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.584998    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:56.589424 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.589405    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-139858168842544"]
Jul 15 20:15:56.667620 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.667537    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shg79\" (UniqueName: \"kubernetes.io/projected/6214b0a5-60b7-4255-82ec-25ee8bb5649d-kube-api-access-shg79\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:15:56.766087 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.766048    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"]
Jul 15 20:15:56.766502 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.766485    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" containerName="test-curl-7613-139858168842544"
Jul 15 20:15:56.766502 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.766502    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" containerName="test-curl-7613-139858168842544"
Jul 15 20:15:56.766652 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.766588    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" containerName="test-curl-7613-139858168842544"
Jul 15 20:15:56.769788 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.769765    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:15:56.772703 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.772675    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd4br\""
Jul 15 20:15:56.776574 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.776551    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"]
Jul 15 20:15:56.869579 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.869539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ae3ab-a1fb-419e-8828-b3974d07c855-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7\" (UID: \"c54ae3ab-a1fb-419e-8828-b3974d07c855\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:15:56.971269 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.971241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ae3ab-a1fb-419e-8828-b3974d07c855-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7\" (UID: \"c54ae3ab-a1fb-419e-8828-b3974d07c855\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:15:56.971834 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:56.971810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ae3ab-a1fb-419e-8828-b3974d07c855-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7\" (UID: \"c54ae3ab-a1fb-419e-8828-b3974d07c855\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:15:57.082207 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.082171    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:15:57.222974 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.222945    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"]
Jul 15 20:15:57.223757 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:15:57.223723    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54ae3ab_a1fb_419e_8828_b3974d07c855.slice/crio-a5a33f8b8b215548143e017829964966a0b8a0801f1c71d7ed83839e5e87222f WatchSource:0}: Error finding container a5a33f8b8b215548143e017829964966a0b8a0801f1c71d7ed83839e5e87222f: Status 404 returned error can't find the container with id a5a33f8b8b215548143e017829964966a0b8a0801f1c71d7ed83839e5e87222f
Jul 15 20:15:57.402398 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.402368    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:57.402577 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.402408    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="427d78cbf2ff8d7048c2421876fa2615996e9ac81efdc1eb21fdbb8bcf4fcb33"
Jul 15 20:15:57.403726 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.403698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7" event={"ID":"c54ae3ab-a1fb-419e-8828-b3974d07c855","Type":"ContainerStarted","Data":"a5a33f8b8b215548143e017829964966a0b8a0801f1c71d7ed83839e5e87222f"}
Jul 15 20:15:57.411938 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:57.411906    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168842544\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:57.455924 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.455892    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:57.557275 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:57.557253    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:15:58.410821 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:58.410783    2564 generic.go:356] "Generic (PLEG): container finished" podID="c54ae3ab-a1fb-419e-8828-b3974d07c855" containerID="ecb476e68effff54107fa18f1aa4caac8406affbc5f54ce0e7bcaa22284bdaf3" exitCode=0
Jul 15 20:15:58.410993 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:58.410867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7" event={"ID":"c54ae3ab-a1fb-419e-8828-b3974d07c855","Type":"ContainerDied","Data":"ecb476e68effff54107fa18f1aa4caac8406affbc5f54ce0e7bcaa22284bdaf3"}
Jul 15 20:15:58.412978 ip-10-0-130-66 kubenswrapper[2564]: E0715 20:15:58.412953    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-139858168842544\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" pod="opendatahub/test-curl-7613-139858168842544"
Jul 15 20:15:58.493342 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:58.493308    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6214b0a5-60b7-4255-82ec-25ee8bb5649d" path="/var/lib/kubelet/pods/6214b0a5-60b7-4255-82ec-25ee8bb5649d/volumes"
Jul 15 20:15:59.559061 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:59.559034    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:15:59.598564 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:59.598530    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ae3ab-a1fb-419e-8828-b3974d07c855-maas-api-service-ca\") pod \"c54ae3ab-a1fb-419e-8828-b3974d07c855\" (UID: \"c54ae3ab-a1fb-419e-8828-b3974d07c855\") "
Jul 15 20:15:59.599084 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:59.598918    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54ae3ab-a1fb-419e-8828-b3974d07c855-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c54ae3ab-a1fb-419e-8828-b3974d07c855" (UID: "c54ae3ab-a1fb-419e-8828-b3974d07c855"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:15:59.700443 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:15:59.700358    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c54ae3ab-a1fb-419e-8828-b3974d07c855-maas-api-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:16:00.423460 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:00.423428    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7"
Jul 15 20:16:00.423672 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:00.423459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7" event={"ID":"c54ae3ab-a1fb-419e-8828-b3974d07c855","Type":"ContainerDied","Data":"a5a33f8b8b215548143e017829964966a0b8a0801f1c71d7ed83839e5e87222f"}
Jul 15 20:16:00.423672 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:00.423515    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5a33f8b8b215548143e017829964966a0b8a0801f1c71d7ed83839e5e87222f"
Jul 15 20:16:07.219385 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:07.219356    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:07.226956 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:07.226928    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"]
Jul 15 20:16:07.229571 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:07.229549    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-826a0f-29735775-74n4q"]
Jul 15 20:16:08.492793 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:08.492757    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c84c7e17-4971-4089-a67b-6e4ef2f2ec10" path="/var/lib/kubelet/pods/c84c7e17-4971-4089-a67b-6e4ef2f2ec10/volumes"
Jul 15 20:16:11.864353 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:11.864312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:12.285802 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:12.285769    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:18.154942 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:18.154910    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:18.214804 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:18.214763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:18.431406 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:18.431314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:18.670444 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:18.670416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:19.414846 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:19.414812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:19.511518 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:19.511484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:29.231487 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.231401    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"]
Jul 15 20:16:29.231908 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.231670    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k" podUID="80ad1e76-2889-4a8a-9654-2f815582ac3e" containerName="maas-api" containerID="cri-o://6232d12c5ed11d767e3f54b1cc9df7c64d057249e430e8a2f3ee435a06bd4d70" gracePeriod=30
Jul 15 20:16:29.260831 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.260803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:29.261803 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.261750    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"]
Jul 15 20:16:29.264685 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.264659    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b72424-29735775-97snq"]
Jul 15 20:16:29.610336 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.610298    2564 generic.go:356] "Generic (PLEG): container finished" podID="80ad1e76-2889-4a8a-9654-2f815582ac3e" containerID="6232d12c5ed11d767e3f54b1cc9df7c64d057249e430e8a2f3ee435a06bd4d70" exitCode=0
Jul 15 20:16:29.610519 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.610373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k" event={"ID":"80ad1e76-2889-4a8a-9654-2f815582ac3e","Type":"ContainerDied","Data":"6232d12c5ed11d767e3f54b1cc9df7c64d057249e430e8a2f3ee435a06bd4d70"}
Jul 15 20:16:29.696405 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.696380    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:16:29.873197 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.873106    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq5nj\" (UniqueName: \"kubernetes.io/projected/80ad1e76-2889-4a8a-9654-2f815582ac3e-kube-api-access-vq5nj\") pod \"80ad1e76-2889-4a8a-9654-2f815582ac3e\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") "
Jul 15 20:16:29.873197 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.873157    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ad1e76-2889-4a8a-9654-2f815582ac3e-openshift-service-ca\") pod \"80ad1e76-2889-4a8a-9654-2f815582ac3e\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") "
Jul 15 20:16:29.873424 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.873202    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ad1e76-2889-4a8a-9654-2f815582ac3e-maas-api-tls\") pod \"80ad1e76-2889-4a8a-9654-2f815582ac3e\" (UID: \"80ad1e76-2889-4a8a-9654-2f815582ac3e\") "
Jul 15 20:16:29.873555 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.873526    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ad1e76-2889-4a8a-9654-2f815582ac3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80ad1e76-2889-4a8a-9654-2f815582ac3e" (UID: "80ad1e76-2889-4a8a-9654-2f815582ac3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:16:29.875501 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.875472    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ad1e76-2889-4a8a-9654-2f815582ac3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80ad1e76-2889-4a8a-9654-2f815582ac3e" (UID: "80ad1e76-2889-4a8a-9654-2f815582ac3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:16:29.875629 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.875548    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ad1e76-2889-4a8a-9654-2f815582ac3e-kube-api-access-vq5nj" (OuterVolumeSpecName: "kube-api-access-vq5nj") pod "80ad1e76-2889-4a8a-9654-2f815582ac3e" (UID: "80ad1e76-2889-4a8a-9654-2f815582ac3e"). InnerVolumeSpecName "kube-api-access-vq5nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:16:29.974243 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.974177    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq5nj\" (UniqueName: \"kubernetes.io/projected/80ad1e76-2889-4a8a-9654-2f815582ac3e-kube-api-access-vq5nj\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:16:29.974243 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.974213    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ad1e76-2889-4a8a-9654-2f815582ac3e-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:16:29.974243 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:29.974253    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ad1e76-2889-4a8a-9654-2f815582ac3e-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Jul 15 20:16:30.494081 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:30.494049    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f99189-81dc-4cb9-90af-deade6fec341" path="/var/lib/kubelet/pods/c3f99189-81dc-4cb9-90af-deade6fec341/volumes"
Jul 15 20:16:30.617981 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:30.617950    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"
Jul 15 20:16:30.617981 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:30.617976    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k" event={"ID":"80ad1e76-2889-4a8a-9654-2f815582ac3e","Type":"ContainerDied","Data":"27407fb1ad37cf965bcde1737e07a7b6df52496caef9638763c73f7b626c1097"}
Jul 15 20:16:30.618243 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:30.618037    2564 scope.go:117] "RemoveContainer" containerID="6232d12c5ed11d767e3f54b1cc9df7c64d057249e430e8a2f3ee435a06bd4d70"
Jul 15 20:16:30.635268 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:30.635241    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"]
Jul 15 20:16:30.637880 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:30.637861    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b72424-ff5cf9954-n8p8k"]
Jul 15 20:16:32.493634 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:32.493589    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ad1e76-2889-4a8a-9654-2f815582ac3e" path="/var/lib/kubelet/pods/80ad1e76-2889-4a8a-9654-2f815582ac3e/volumes"
Jul 15 20:16:33.888144 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:33.888108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:34.349348 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:34.349304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:40.368510 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:40.368477    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:40.424362 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:40.424330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:40.663563 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:40.663473    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:40.908334 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:40.908299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:16:41.365262 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:41.365215    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cc87cffbc-kqljb_b2367c66-2919-421e-accb-2dda4bc0d13b/authorino/0.log"
Jul 15 20:16:47.374896 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:47.374861    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c87d8b7bd-vcmtc_a68340ee-cf3d-4681-86de-eee569d7d52d/manager/0.log"
Jul 15 20:16:47.860585 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:47.860550    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58dd997468-wwljd_989a7f1f-f58c-4170-863c-5e2a2802667a/manager/0.log"
Jul 15 20:16:47.866481 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:47.866459    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58dd997468-wwljd_989a7f1f-f58c-4170-863c-5e2a2802667a/copy-manifests/0.log"
Jul 15 20:16:48.404466 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:48.404439    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7874567466-rxfft_f79769cd-b7c1-4ce6-b9b7-f0c05d58ceaa/maas-api/0.log"
Jul 15 20:16:48.529349 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:48.529320    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735745-8tc6d_f82b629c-d912-47aa-8df6-b8df0fb70d54/cleanup/0.log"
Jul 15 20:16:48.646611 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:48.646580    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735760-46h2b_cd19f1be-7d65-4e89-bbce-7251014e7e37/cleanup/0.log"
Jul 15 20:16:48.768029 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:48.768003    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735775-6k47d_6157ccf2-360a-40c5-a80d-d0c22eb040c6/cleanup/0.log"
Jul 15 20:16:48.883181 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:48.883150    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-37d9d895-p9h6z_44edf938-8e57-4e29-b157-68ef793494d9/revoke-keys/0.log"
Jul 15 20:16:48.996204 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:48.996170    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-de95558c-qt8t9_8ca5214c-aa01-4e2f-a271-406909678ae3/revoke-keys/0.log"
Jul 15 20:16:49.376014 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:49.375987    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-033c5f64-ldm52_56d431f1-1a55-406c-912e-83c39067fa75/revoke-keys/0.log"
Jul 15 20:16:49.610000 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:49.609971    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-c9860b-hh8s2_2ef7b513-036c-4f41-9c35-72d93f860023/revoke-keys/0.log"
Jul 15 20:16:49.725070 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:49.725043    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-2791e51d-twnsj_2a8de539-0b55-4235-8c95-1e299e3e5dbc/revoke-keys/0.log"
Jul 15 20:16:49.846042 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:49.846010    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-12e2475c-h5zc5_52009507-a6fe-47c4-bc39-ef9207719ef6/revoke-keys/0.log"
Jul 15 20:16:49.973075 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:49.973047    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-3a3ed256-klvrj_667caec5-1e01-4089-9064-5901c9236590/revoke-keys/0.log"
Jul 15 20:16:50.091085 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:50.091017    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-826a0f-b8sl7_c54ae3ab-a1fb-419e-8828-b3974d07c855/revoke-keys/0.log"
Jul 15 20:16:50.321543 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:50.321511    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-q68xn_254ac940-9cf6-4025-aa45-117c82689fa1/postgres/0.log"
Jul 15 20:16:51.145500 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.145476    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb_1cce0045-1991-47dd-83ca-3bcde525eb79/util/0.log"
Jul 15 20:16:51.150733 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.150710    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb_1cce0045-1991-47dd-83ca-3bcde525eb79/pull/0.log"
Jul 15 20:16:51.156001 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.155980    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb_1cce0045-1991-47dd-83ca-3bcde525eb79/extract/0.log"
Jul 15 20:16:51.282149 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.282121    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf_45ea731d-620a-4baf-ab42-e55e205c6d2f/util/0.log"
Jul 15 20:16:51.288632 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.288608    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf_45ea731d-620a-4baf-ab42-e55e205c6d2f/pull/0.log"
Jul 15 20:16:51.295027 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.295003    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf_45ea731d-620a-4baf-ab42-e55e205c6d2f/extract/0.log"
Jul 15 20:16:51.414212 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.414139    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g_cdfd402c-ee4e-4438-8b61-12d00cfeee2e/util/0.log"
Jul 15 20:16:51.420542 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.420519    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g_cdfd402c-ee4e-4438-8b61-12d00cfeee2e/pull/0.log"
Jul 15 20:16:51.427628 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.427610    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g_cdfd402c-ee4e-4438-8b61-12d00cfeee2e/extract/0.log"
Jul 15 20:16:51.544013 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.543983    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d_dd4266d3-d64c-4aa7-8862-ec2f2f078be3/util/0.log"
Jul 15 20:16:51.550163 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.550139    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d_dd4266d3-d64c-4aa7-8862-ec2f2f078be3/pull/0.log"
Jul 15 20:16:51.556387 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.556366    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d_dd4266d3-d64c-4aa7-8862-ec2f2f078be3/extract/0.log"
Jul 15 20:16:51.682112 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:51.682035    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cc87cffbc-kqljb_b2367c66-2919-421e-accb-2dda4bc0d13b/authorino/0.log"
Jul 15 20:16:53.150537 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:53.150500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b66bc9f5-hpnjw_18631460-5a0f-4b5c-b7c8-92e09620ca63/kube-auth-proxy/0.log"
Jul 15 20:16:53.788884 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:53.788856    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9ccbbdb-bqbnh_299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2/router/0.log"
Jul 15 20:16:54.169344 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:16:54.169209    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nzwnd_62cb5290-a72a-4440-bf52-d31d74a9516b/main/0.log"
Jul 15 20:17:01.297586 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.297541    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7q79m/must-gather-fbt77"]
Jul 15 20:17:01.297986 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.297930    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54ae3ab-a1fb-419e-8828-b3974d07c855" containerName="revoke-keys"
Jul 15 20:17:01.297986 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.297942    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54ae3ab-a1fb-419e-8828-b3974d07c855" containerName="revoke-keys"
Jul 15 20:17:01.297986 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.297955    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ad1e76-2889-4a8a-9654-2f815582ac3e" containerName="maas-api"
Jul 15 20:17:01.297986 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.297962    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ad1e76-2889-4a8a-9654-2f815582ac3e" containerName="maas-api"
Jul 15 20:17:01.298115 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.298028    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54ae3ab-a1fb-419e-8828-b3974d07c855" containerName="revoke-keys"
Jul 15 20:17:01.298115 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.298037    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ad1e76-2889-4a8a-9654-2f815582ac3e" containerName="maas-api"
Jul 15 20:17:01.300655 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.300624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:17:01.301234 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.301202    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.303377 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.303361    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7q79m\"/\"openshift-service-ca.crt\""
Jul 15 20:17:01.304082 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.304060    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7q79m\"/\"default-dockercfg-ptx96\""
Jul 15 20:17:01.304190 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.304132    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7q79m\"/\"kube-root-ca.crt\""
Jul 15 20:17:01.311704 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.311677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7q79m/must-gather-fbt77"]
Jul 15 20:17:01.399095 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.399017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log" limit=1024
Jul 15 20:17:01.460495 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.460454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e97f6886-e8f8-4da3-a504-1fd70dccf4d5-must-gather-output\") pod \"must-gather-fbt77\" (UID: \"e97f6886-e8f8-4da3-a504-1fd70dccf4d5\") " pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.460674 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.460545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9qq9\" (UniqueName: \"kubernetes.io/projected/e97f6886-e8f8-4da3-a504-1fd70dccf4d5-kube-api-access-n9qq9\") pod \"must-gather-fbt77\" (UID: \"e97f6886-e8f8-4da3-a504-1fd70dccf4d5\") " pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.562053 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.561949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e97f6886-e8f8-4da3-a504-1fd70dccf4d5-must-gather-output\") pod \"must-gather-fbt77\" (UID: \"e97f6886-e8f8-4da3-a504-1fd70dccf4d5\") " pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.562053 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.562039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9qq9\" (UniqueName: \"kubernetes.io/projected/e97f6886-e8f8-4da3-a504-1fd70dccf4d5-kube-api-access-n9qq9\") pod \"must-gather-fbt77\" (UID: \"e97f6886-e8f8-4da3-a504-1fd70dccf4d5\") " pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.562373 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.562353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e97f6886-e8f8-4da3-a504-1fd70dccf4d5-must-gather-output\") pod \"must-gather-fbt77\" (UID: \"e97f6886-e8f8-4da3-a504-1fd70dccf4d5\") " pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.569680 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.569645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9qq9\" (UniqueName: \"kubernetes.io/projected/e97f6886-e8f8-4da3-a504-1fd70dccf4d5-kube-api-access-n9qq9\") pod \"must-gather-fbt77\" (UID: \"e97f6886-e8f8-4da3-a504-1fd70dccf4d5\") " pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.613316 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.613285    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7q79m/must-gather-fbt77"
Jul 15 20:17:01.750520 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.750488    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7q79m/must-gather-fbt77"]
Jul 15 20:17:01.751905 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:17:01.751885    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode97f6886_e8f8_4da3_a504_1fd70dccf4d5.slice/crio-064bec77804d26611c5c0a8ac92d6cb977054128d86d5f68de7375651405e49b WatchSource:0}: Error finding container 064bec77804d26611c5c0a8ac92d6cb977054128d86d5f68de7375651405e49b: Status 404 returned error can't find the container with id 064bec77804d26611c5c0a8ac92d6cb977054128d86d5f68de7375651405e49b
Jul 15 20:17:01.816525 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:01.816438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7q79m/must-gather-fbt77" event={"ID":"e97f6886-e8f8-4da3-a504-1fd70dccf4d5","Type":"ContainerStarted","Data":"064bec77804d26611c5c0a8ac92d6cb977054128d86d5f68de7375651405e49b"}
Jul 15 20:17:03.834163 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:03.834124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7q79m/must-gather-fbt77" event={"ID":"e97f6886-e8f8-4da3-a504-1fd70dccf4d5","Type":"ContainerStarted","Data":"60633cf689092734be092ab6f85073b42f9d54fe483859f0fe9734d6ecdfdeec"}
Jul 15 20:17:03.834778 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:03.834731    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7q79m/must-gather-fbt77" event={"ID":"e97f6886-e8f8-4da3-a504-1fd70dccf4d5","Type":"ContainerStarted","Data":"14046698cc36a797821ded02afa4e5ee8d318d8780a00f38e10e9a13d4494cae"}
Jul 15 20:17:03.852473 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:03.852415    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7q79m/must-gather-fbt77" podStartSLOduration=1.735687198 podStartE2EDuration="2.852394276s" totalImagesPullingTime="1.116707078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:17:03.84867366 +0000 UTC m=+2913.926466461" watchObservedRunningTime="2026-07-15 20:17:03.852394276 +0000 UTC m=+2913.930187078"
Jul 15 20:17:04.423348 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:04.423317    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2z2gr_dbb785a9-ed6f-4480-bd9c-e190191eaf8e/global-pull-secret-syncer/0.log"
Jul 15 20:17:04.548410 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:04.548384    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q9675_30e421da-72b9-4111-afc5-cfcbb4df4731/konnectivity-agent/0.log"
Jul 15 20:17:04.627284 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:04.627248    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-66.ec2.internal_54943181b010dce43fa747b7f7cd23b0/haproxy/0.log"
Jul 15 20:17:08.915424 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:08.915389    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb_1cce0045-1991-47dd-83ca-3bcde525eb79/extract/0.log"
Jul 15 20:17:08.946281 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:08.946247    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb_1cce0045-1991-47dd-83ca-3bcde525eb79/util/0.log"
Jul 15 20:17:08.975671 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:08.975595    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d8krb_1cce0045-1991-47dd-83ca-3bcde525eb79/pull/0.log"
Jul 15 20:17:09.005824 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.005757    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf_45ea731d-620a-4baf-ab42-e55e205c6d2f/extract/0.log"
Jul 15 20:17:09.030531 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.030503    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf_45ea731d-620a-4baf-ab42-e55e205c6d2f/util/0.log"
Jul 15 20:17:09.057309 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.057283    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wdkhf_45ea731d-620a-4baf-ab42-e55e205c6d2f/pull/0.log"
Jul 15 20:17:09.084273 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.084248    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g_cdfd402c-ee4e-4438-8b61-12d00cfeee2e/extract/0.log"
Jul 15 20:17:09.105007 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.104983    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g_cdfd402c-ee4e-4438-8b61-12d00cfeee2e/util/0.log"
Jul 15 20:17:09.128387 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.128356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73llp2g_cdfd402c-ee4e-4438-8b61-12d00cfeee2e/pull/0.log"
Jul 15 20:17:09.169753 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.169668    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d_dd4266d3-d64c-4aa7-8862-ec2f2f078be3/extract/0.log"
Jul 15 20:17:09.192865 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.192820    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d_dd4266d3-d64c-4aa7-8862-ec2f2f078be3/util/0.log"
Jul 15 20:17:09.219272 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.219246    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ccc5d_dd4266d3-d64c-4aa7-8862-ec2f2f078be3/pull/0.log"
Jul 15 20:17:09.807575 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:09.807349    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cc87cffbc-kqljb_b2367c66-2919-421e-accb-2dda4bc0d13b/authorino/0.log"
Jul 15 20:17:11.959006 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:11.958958    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/alertmanager/0.log"
Jul 15 20:17:11.984645 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:11.984613    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/config-reloader/0.log"
Jul 15 20:17:12.008162 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.008113    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/kube-rbac-proxy-web/0.log"
Jul 15 20:17:12.032517 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.032427    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/kube-rbac-proxy/0.log"
Jul 15 20:17:12.053296 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.053261    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/kube-rbac-proxy-metric/0.log"
Jul 15 20:17:12.110614 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.110589    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/prom-label-proxy/0.log"
Jul 15 20:17:12.165543 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.165414    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b028c2e6-aea2-47ef-a8f4-f1fe0d1b7fc1/init-config-reloader/0.log"
Jul 15 20:17:12.322845 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.322786    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8545cf9cdb-tvdjz_4313cff1-9090-46cc-8186-0bed2a37c32e/metrics-server/0.log"
Jul 15 20:17:12.535679 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.535620    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vhddg_036e2a6c-b057-44ce-9778-0540d4f5d4a2/node-exporter/0.log"
Jul 15 20:17:12.556360 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.556318    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vhddg_036e2a6c-b057-44ce-9778-0540d4f5d4a2/kube-rbac-proxy/0.log"
Jul 15 20:17:12.576360 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.576329    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vhddg_036e2a6c-b057-44ce-9778-0540d4f5d4a2/init-textfile/0.log"
Jul 15 20:17:12.608322 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.608284    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s9tvz_a7fbabbe-3115-4f5d-b281-66a1e05c5a27/kube-rbac-proxy-main/0.log"
Jul 15 20:17:12.630178 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.630132    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s9tvz_a7fbabbe-3115-4f5d-b281-66a1e05c5a27/kube-rbac-proxy-self/0.log"
Jul 15 20:17:12.651185 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.651156    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-s9tvz_a7fbabbe-3115-4f5d-b281-66a1e05c5a27/openshift-state-metrics/0.log"
Jul 15 20:17:12.867892 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.867788    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-cv4rf_d2ec833a-e136-4858-b688-9eed5924af6c/prometheus-operator/0.log"
Jul 15 20:17:12.884703 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.884670    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-cv4rf_d2ec833a-e136-4858-b688-9eed5924af6c/kube-rbac-proxy/0.log"
Jul 15 20:17:12.911275 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.911213    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-j8ghn_3bc3332d-3937-4e17-af38-cb06e98059f5/prometheus-operator-admission-webhook/0.log"
Jul 15 20:17:12.942598 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.942547    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-686dff948d-fhvcd_fe4d0f7b-727a-4746-83e8-4822fc359b09/telemeter-client/0.log"
Jul 15 20:17:12.962634 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.962606    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-686dff948d-fhvcd_fe4d0f7b-727a-4746-83e8-4822fc359b09/reload/0.log"
Jul 15 20:17:12.983854 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:12.983820    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-686dff948d-fhvcd_fe4d0f7b-727a-4746-83e8-4822fc359b09/kube-rbac-proxy/0.log"
Jul 15 20:17:13.020414 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:13.020362    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cc6f8fcb4-f5vsz_d795ccaf-767d-4335-909b-8fcbb1ebbea8/thanos-query/0.log"
Jul 15 20:17:13.039962 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:13.039937    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cc6f8fcb4-f5vsz_d795ccaf-767d-4335-909b-8fcbb1ebbea8/kube-rbac-proxy-web/0.log"
Jul 15 20:17:13.060341 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:13.060315    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cc6f8fcb4-f5vsz_d795ccaf-767d-4335-909b-8fcbb1ebbea8/kube-rbac-proxy/0.log"
Jul 15 20:17:13.080164 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:13.080121    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cc6f8fcb4-f5vsz_d795ccaf-767d-4335-909b-8fcbb1ebbea8/prom-label-proxy/0.log"
Jul 15 20:17:13.100061 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:13.100026    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cc6f8fcb4-f5vsz_d795ccaf-767d-4335-909b-8fcbb1ebbea8/kube-rbac-proxy-rules/0.log"
Jul 15 20:17:13.119211 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:13.119128    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cc6f8fcb4-f5vsz_d795ccaf-767d-4335-909b-8fcbb1ebbea8/kube-rbac-proxy-metrics/0.log"
Jul 15 20:17:14.968100 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:14.968065    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"]
Jul 15 20:17:14.977425 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:14.977396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:14.979993 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:14.979962    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"]
Jul 15 20:17:15.016706 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.016666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-sys\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.016938 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.016921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-proc\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.017093 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.017077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-lib-modules\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.017253 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.017237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4lln\" (UniqueName: \"kubernetes.io/projected/212bfa6d-a731-4ac8-bad3-a21f98202530-kube-api-access-x4lln\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.017436 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.017418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-podres\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119016    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4lln\" (UniqueName: \"kubernetes.io/projected/212bfa6d-a731-4ac8-bad3-a21f98202530-kube-api-access-x4lln\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-podres\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-sys\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-proc\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-lib-modules\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-lib-modules\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-podres\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119834    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-sys\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.119906 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.119879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/212bfa6d-a731-4ac8-bad3-a21f98202530-proc\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.130042 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.130012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4lln\" (UniqueName: \"kubernetes.io/projected/212bfa6d-a731-4ac8-bad3-a21f98202530-kube-api-access-x4lln\") pod \"perf-node-gather-daemonset-hbgbw\" (UID: \"212bfa6d-a731-4ac8-bad3-a21f98202530\") " pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.292452 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.292364    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.482486 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.482458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"]
Jul 15 20:17:15.485492 ip-10-0-130-66 kubenswrapper[2564]: W0715 20:17:15.485447    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod212bfa6d_a731_4ac8_bad3_a21f98202530.slice/crio-cea8719da4ba3d1bd154b52462e3f7e0b962727e8bf9e6a7fbd39f4423cd3100 WatchSource:0}: Error finding container cea8719da4ba3d1bd154b52462e3f7e0b962727e8bf9e6a7fbd39f4423cd3100: Status 404 returned error can't find the container with id cea8719da4ba3d1bd154b52462e3f7e0b962727e8bf9e6a7fbd39f4423cd3100
Jul 15 20:17:15.536511 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.536457    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-556b9c656b-hf7qp_f5dbbd7e-e131-45e0-9bf4-8c01a08f36ad/console/0.log"
Jul 15 20:17:15.570763 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.570728    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-d7hbg_b05d4d84-59fb-4855-83d2-5f285ffcea40/download-server/0.log"
Jul 15 20:17:15.969058 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.969014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw" event={"ID":"212bfa6d-a731-4ac8-bad3-a21f98202530","Type":"ContainerStarted","Data":"496d062d1f5f83c4b4d1e12265c737bda5bfc9a33f76cb37c20cb0c7d5e8bb58"}
Jul 15 20:17:15.969569 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.969086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw" event={"ID":"212bfa6d-a731-4ac8-bad3-a21f98202530","Type":"ContainerStarted","Data":"cea8719da4ba3d1bd154b52462e3f7e0b962727e8bf9e6a7fbd39f4423cd3100"}
Jul 15 20:17:15.969974 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.969949    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:15.985393 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:15.985336    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw" podStartSLOduration=1.985313844 podStartE2EDuration="1.985313844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:17:15.983371495 +0000 UTC m=+2926.061164297" watchObservedRunningTime="2026-07-15 20:17:15.985313844 +0000 UTC m=+2926.063106648"
Jul 15 20:17:17.230358 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:17.230323    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rktvs_d15779c7-6817-42eb-98ad-71fee2675631/dns/0.log"
Jul 15 20:17:17.253112 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:17.253085    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rktvs_d15779c7-6817-42eb-98ad-71fee2675631/kube-rbac-proxy/0.log"
Jul 15 20:17:17.304651 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:17.304621    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jx4q7_4e405546-e58e-4c94-a49f-c6c18b7a28d6/dns-node-resolver/0.log"
Jul 15 20:17:17.897241 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:17.897194    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k29ss_dcb8fe74-76fa-416d-9203-992448ca4ddd/node-ca/0.log"
Jul 15 20:17:18.969499 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:18.969471    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b66bc9f5-hpnjw_18631460-5a0f-4b5c-b7c8-92e09620ca63/kube-auth-proxy/0.log"
Jul 15 20:17:19.129391 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:19.129363    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9ccbbdb-bqbnh_299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2/router/0.log"
Jul 15 20:17:19.857322 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:19.857296    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7pcjt_e628e49a-bcdb-46c6-91d2-e6942e8b1eb5/serve-healthcheck-canary/0.log"
Jul 15 20:17:20.311521 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:20.311491    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-fp5ps_22a24f49-4050-4490-aaaa-0a317701ded1/insights-operator/0.log"
Jul 15 20:17:20.311993 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:20.311972    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-fp5ps_22a24f49-4050-4490-aaaa-0a317701ded1/insights-operator/1.log"
Jul 15 20:17:20.401926 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:20.401878    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6brrv_398023a0-b103-4631-9812-b1785776bfe6/kube-rbac-proxy/0.log"
Jul 15 20:17:20.421710 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:20.421674    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6brrv_398023a0-b103-4631-9812-b1785776bfe6/exporter/0.log"
Jul 15 20:17:20.443782 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:20.443752    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6brrv_398023a0-b103-4631-9812-b1785776bfe6/extractor/0.log"
Jul 15 20:17:22.915948 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:22.915900    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c87d8b7bd-vcmtc_a68340ee-cf3d-4681-86de-eee569d7d52d/manager/0.log"
Jul 15 20:17:23.003553 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:23.003526    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7q79m/perf-node-gather-daemonset-hbgbw"
Jul 15 20:17:23.075625 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:23.075592    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58dd997468-wwljd_989a7f1f-f58c-4170-863c-5e2a2802667a/manager/0.log"
Jul 15 20:17:23.094102 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:23.094077    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58dd997468-wwljd_989a7f1f-f58c-4170-863c-5e2a2802667a/copy-manifests/0.log"
Jul 15 20:17:24.577553 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:24.577524    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7784c86d4c-ffnll_c1d799d6-1c80-4b78-b31f-21c2546b92e2/manager/0.log"
Jul 15 20:17:25.657892 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:25.657859    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b66bc9f5-hpnjw_18631460-5a0f-4b5c-b7c8-92e09620ca63/kube-auth-proxy/0.log"
Jul 15 20:17:25.777368 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:25.777340    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9ccbbdb-bqbnh_299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2/router/0.log"
Jul 15 20:17:27.188004 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:27.187909    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b66bc9f5-hpnjw_18631460-5a0f-4b5c-b7c8-92e09620ca63/kube-auth-proxy/0.log"
Jul 15 20:17:27.314582 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:27.314551    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9ccbbdb-bqbnh_299af2aa-3bc0-4ecf-bee7-639ffdbe6fe2/router/0.log"
Jul 15 20:17:31.038010 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.037980    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9xbbj_8a0c5440-1940-4719-adfb-e10cff36d0f6/kube-multus/0.log"
Jul 15 20:17:31.359776 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.359709    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/kube-multus-additional-cni-plugins/0.log"
Jul 15 20:17:31.380200 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.380174    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/egress-router-binary-copy/0.log"
Jul 15 20:17:31.400662 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.400637    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/cni-plugins/0.log"
Jul 15 20:17:31.419310 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.419285    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/bond-cni-plugin/0.log"
Jul 15 20:17:31.440826 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.440802    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/routeoverride-cni/0.log"
Jul 15 20:17:31.461670 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.461648    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/whereabouts-cni-bincopy/0.log"
Jul 15 20:17:31.479693 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.479672    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztbhl_fcdbed1e-b25a-49f4-8957-e6687fcaf0f5/whereabouts-cni/0.log"
Jul 15 20:17:31.599110 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.599067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9pclt_88f9a415-fea9-462f-83bf-e457ab604332/network-metrics-daemon/0.log"
Jul 15 20:17:31.616094 ip-10-0-130-66 kubenswrapper[2564]: I0715 20:17:31.616029    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9pclt_88f9a415-fea9-462f-83bf-e457ab604332/kube-rbac-proxy/0.log"